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PREFACE 


This  document  is  a  report  prepared  for  the  Rome  Air  Development  Center  (RADC) 
and  the  US  Army  Computer  Systems  Command/AIRMICS  in  support  of  the  Automation 
of  Quality  Measurement  project.  It  is  the  final  report  (CDRL  A003)  for  the 
Contract  No.  F30602-79-C-0267.  The  purpose  of  the  project  was  to  provide 
computer  programs,  supporting  documents,  and  research  results  related  to  the 
effort  of  measuring  certain  quality  characteristics  of  software. 

This  report  was  prepared  by  J.  McCall  and  D.  Markham.  Contributions  were  made 
by  R.  McGindley,  M.  Hegedus,  M.  Matsumoto,  A.  Stone,  and  M.  Stosick. 

Technical  guidance  was  provided  by  Mr.  J.  Cavano  of  RADC  and  supported  by  Mr. 
D.  Hocking  of  AIRMICS. 

The  objective  of  this  study  was  to  establish  and  demonstrate  a  method  of 
automating  the  measurement  of  significant  aspects  of  software  quality. 
Conceptually,  the  method  of  software  measurement  through  metrics  provides  a 
mechanism  in  conjunction  with  a  vigorous  development  program  to  provide 
management  a  technique  to  improve  the  quality  of  software  products. 


SECTION  1 
INTRODUCTION 

1.1  IDENTIFICATION 

This  document  is  the  Final  Report  of  the  research  and  development  tasks 
associated  with  the  development  of  the  Automated  Metrics  Tool  (AMT).  The 
development  and  construction  of  this  prototype  software  tool  was  provided  by 
GE  Western  Systems,  Sunnyvale,  Calif,  in  compliance  with  the  requirements  set 
forth  by  Rome  Air  Development  Center  (RADC)  and  the  US  Army  Computer  Systems 
Command /AIRMICS  for  the  Automation  of  Quality  Measurement  Project,  Contract 
No.  F30602-79-C-0267. 

1.2  SCOPE 

This  document  Includes  a  description  of  the  AMT  and  the  results  of  some 
parallel  research  efforts;  the  use  of  AMT  on  Itself  to  derive  quality 
measurements  of  the  tool,  the  design  goals  of  the  AMT,  the  conversion  of  AMT 
from  a  VAX  11/780  development  machine  to  a  Honeywell  6000  series  host 
environment,  and  the  comparison  of  metric  scores  across  several  projects 
including  the  AMT.  Also  Included  is  a  description  of  how  the  AMT  could  be 
used  to  support  a  software  development  activity. 

1.3  ORGANIZATION  OF  DOCUMENT 

This  section  of  the  document  Is  an  Introduction  to  the  remainder  of  the 
report.  The  second  section  provides  a  brief  description  of  the  background  and 
application  of  metric  concepts  referencing  previously  funded  RADC  and  USACSC 
■unded  research.  The  third  section  will  describe  how  the  AMT  was  developed 
and  the  motivation  for  developing  it.  Section  4  is  a  description  of  the  AMT. 
Section  5  describes  how  we  used  the  AMT  during  Its  development  to  apply 
metrics.  The  Final  Section  suggests  future  research  In  metrics  and  identifies 
enhanced  capabilities  that  should  be  considered  for  the  AMT. 

1.4  APPLICABLE  DOCUMENTS 

The  following  documents  Include  those  published  and  distributed  by  RADC  which 
are  background  to  this  effort  and  explain  In  detail  the  concept  and  manual 
application  of  software  quality  metrics,  as  well  as  those  produced  as  a  result 
of  this  research  task: 
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McCall,  J.A.  et  al.,  "Factors  In  Software  Quality".  RADC-TR-77-369,  June  1977. 

McCall,  J.A.,  and  Matsumoto,  M.T.,  "Metrics  Enhancement  Final  Report", 
RADC-TR-80-109,  Volume  I,  April  1980. 

McCall,  J.A. ,  and  Matsumoto,  M.T.,  "Software  Quality  Measurement  Manual", 
RADC-TR-80-109,  Volume  II,  April  1980. 

The  applicable  documents  produced  during  the  course  of  this  research  task, 
identified  with  their  associated  CDRL  item  number: 


AMT  User's  Manual  A012 
AMT  Training  Material  A006 
AMT  Program  Maintenance  Manual  A013 
AMT  Functional  Description  A007 
AMT  Data  Requirement  Document  A008 
AMT  Sys. /Subsystem  Specification  A009 
AMT  Program  Specification  A010 
AMT  Data  Base  Description  A011 
AMT  Test  Plan  A015 
Test  Analysis  A014 
AMT  Program  Maintenance  Manual  A013 


1.5  EXECUTIVE  SUMMARY 

The  concept  behind  software  quality  metrics  is  to  provide  software  acquisition 
managers  with  a  mechanism  to  quantitatively  specify  and  measure  the  level  of 
quality  in  a  software  product.  To  provide  this  mechanism,  an  acquisition 
manager  or  a  software  developer  must  collect  data  from  the  products  of  the 
software  development  process.  The  actual  data  items  collected  are  Identified 
in  detail  in  the  "Metric  Enhancement  Final  Report",  RADC  TR-80-109.  This  raw 
data  is  then  used  to  calculate  metric  values  which  can  be  used  to  assess  the 
quality  of  the  software  being  produced. 

The  purpose  of  the  Automated  Measurement  Tool  (AMT)  Is  to  provide  automated 
support  to  the  application  of  the  metrics  concept.  Normally,  the  metrics  data 
must  be  collected  by  hand  In  a  tedious,  error-prone,  and  time-consuming 
process. 


The  intent  of  the  AMT  is  to  automatically  collect  and  store  the  metric  data 
economically  and  reliably  and  provide  a  data  base  of  metrics  data  to 
facilitate  research  and  evaluation.  This  will  then  allow  the  acquisition 
manager  to  easily  collect  and  analyze  the  software  quality  metric  values  for 
any  given  software  development  using  the  AMT.  The  AMT  data  base  and  reporting 
capabilities  were  used  to  the  extent  possible  ouring  the  development  of  the 
AMT  itself  to  support  application  of  metrics.  This  is  the  first  actual 
contractual  application  of  metrics  and  the  lessons  learned  from  this 
experience  are  documented  in  this  report. 

The  current  version  of  the  AMT  operates  on  the  Honeywell  6180/GC0S  computer 
system  at  RADC.  It  processes  COBOL  code. 


SECTION  2 

BACKGROUND  OF  SOFTWARE  METRICS 


The  basic  concepts  for  the  software  metrics  automatically  collected, 
calculated,  and  reported  by  the  AMT  were  derived  during  the  Factors  in 
Software  Quality  contract,  contract  number  F30602-76-C-0417.  A  framework  for 
defining  metrics  [CAVJ78],  applying  them  to  all  of  the  products,  including 
documentation,  of  a  software  development,  and  relating  them  to  management 
goals  was  developed  [MCCJ77I].  A  preliminary  handbook  for  an  Air  Force  System 
Program  Office  was  developed  to  describe  the  framework  [MCCJ77III].  Initial 
validation  of  some  of  the  metrics  was  performed  using  command  and  control 
software  systems  written  in  JOVIAL  that  had  2-4  years  of  operation  and 
maintenance  data  available  [MCCJ77II].  During  a  subsequent  research  effort, 
the  Metrics  Enhancement  Contract,  contract  number  F30602-78-C-0216,  further 
validation  was  conducted  using  an  Army  financial  management  information  system 
written  in  COBOL  that  had  been  transported  to  two  different  vendor's  computers 
besides  the  initial  development  system  and  a  software  support  system  written 
in  FORTRAN  that  had  been  transported  to  a  number  of  different  DEC  operating 
systems  as  well  as  a  Honeywell  6000  computer  system.  At  the  end  of  this 
effort,  validation  of  metrics  related  to  the  quality  factors  Reliability, 
Maintainability,  Flexibility,  and  Portablility  was  achieved  [MCCJ79I],  More 
importantly,  techniques  were  developed  to  apply  the  metrics  and  derive 
information  during  a  software  development  that  facilitated  identification  of 
potential  quality  problems,  areas  needing  improvement  in  standards  and 
conventions,  test  strategy,  and  acceptance  criteria.  An  overview  of  these 
techniques  was  provided  in  a  Software  Quality  Measurement  Manual  [MCCJ78II]. 
Currently,  software  metrics  is  one  of  the  most  widely  investigated  subject 
areas  in  the  software  research  community.  Many  individuals  and  organizations 
are  developing  metrics  related  to  or  extending  the  metrics  developed  under  the 
previously  mentioned  RADC/USACSC  funded  research  and  the  pioneering  work  of 
the  others  ([HALM77],  [MCCT76],  [BOEB73],  [CHER],  [FAGM76],  [F0SL76], 
[HESC77]).  The  significance  of  this  continued*  research  is  not  only  the 
refinement  of  the  set  of  metrics  which  can  be  utilized  in  the  framework 
established  in  the  report,  "Factors  in  Software  Quality"  [MCCJ77],  but  also 
the  industry  wide  experiences  being  gained  in  applying  and  using  metrics 
during  software  developments. 
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As  more  organizations  apply  metrics,  more  quantitative  information  is  becoming 
available  about  the  software  being  developed.  This  information  supports 
research  in  other  software  engineering  disciplines  such  as  software  tools  and 
development  environments,  programming  languages,  design  techniques,  and  cost 
estimation. 

It  is  expected  that  the  use  of  software  metrics  will  become  a  standard 
contractual  instrument  to  assure  a  certain  level  of  quality. 

This  growing  recognition  by  government  organizations,  further  refinement  by 
the  research  community,  and  application  experience  by  a  number  of 
organizations  makes  the  introduction  of  a  tool  like  the  AMT  timely. 


SECTION  3 

DEVELOPMENT  OF  THE  AMT 


3.1  CONDUCT  OF  THE  PROJECT 

The  AMT  project  was  conducted  In  four  phases:  design.  Implementation, 
-estlng,  and  dell very/training.  Our  motivation  and  approach  to  each  phase 
will  be  discussed  In  this  section. 

3.2  NEED  FOR  AUTOMATION 

At  the  outset  of  the  project,  a  specific  need  was  envisioned  for  a  family  of 
software  tools  that  would  support  the  specification,  measurement,  and 
reporting  of  software  quality  metrics.  The  viability  of  effective  measurement 
of  software  quality  has  been  enhanced  by  the  evolution  during  the  past  decade 
of  modern  programming  practices,  structured,  disciplined  development 

techniques  and  methodologies,  and  requirements  for  more  structured,  effective 
documentation. 

The  actual  measurement  of  software  quality  Is  accomplished  by  applying 

software  metrics  (or  measurements)  to  the  documentation  and  source  code 
produced  during  a  software  development.  These  measurements  are  part  of  the 
established  model  of  software  quality  and  through  that  model  can  be  related  to 
various  user-oriented  aspects  of  software  quality. 

The  current  set  of  metrics  utilized  In  the  model  which  Is  comprised  of  11 
quality  factors  has  39  software  metrics.  Subsets  of  these  39  metrics  can  be 
applied  during  each  phase  of  the  development.  The  breakdown  by  phase  Is: 

15  can  be  applied  during  requirements 
34  can  be  applied  during  design 
38  can  be  applied  during  Implementation 

To  calculate  this  entire  complement  of  metrics,  296  Individual  data  Items  have 
to  be  collected.  Worksheets,  described  In  paragraph  3.2,  contain  all  of  the 
Individual  data  Items.  A  breakdown  by  phase  of  the  Individual  data  Items  Is: 


31  measured  during  requirements 
173  measured  during  design 
92  measured  during  Implementation 
296  Total 

All  of  the  data  Items  collected  during  requirements  and  102  of  the  items 
collected  during  design  are  system  level  measurements  which  are  taken  once. 
However,  the  remaining  71  Items  collected  during  design  and  the  92  collected 
during  implementation  are  module  level  measurements  which  are  taken  for  each 
module  in  the  system.  Thus,  for  even  a  modestly  sized  system  of  100  modules, 
the  number  of  data  items  to  be  collected  is: 

31  (requirements)  +  102  (design)  +  100x71  (design) 

+100x92  (Implementation)  *  16,433. 

On  a  particular  development  project,  a  subset  of  these  data  Items  relating  to 
the  important  quality  factors  to  that  development  would  be  collected. 

To  collect  this  large  a  number  of  data  items  manually  from  documentation  and 
source  code  is  a  very  time-consuming,  error  prone,  and  thus  expensive  process. 

The  time  consuming  nature  of  manual  Inspection  of  source  code  Impacts  one  of 
the  Important  necessary  conditions  for  the  quality  assurance  effort.  For 
project  management  to  make  the  necessary  corrections  to  meet  a  specified 
quality  goal,  software  quality  assurance  analysts  must  be  able  to  report 
discrepancies  as  soon  as  possible  after  the  code  Is  ready  for  Inspection. 

Metric  data  also  needs  to  be  archived  for  developmental,  life-cycle  and 
research  reasons.  Software  quality  assurance  analysts  need  to  have  an  overall 
view  of  the  metric  scores.  Researchers  need  to  have  a  historical  record 
across  projects  for  comparative  purposes.  This  historical  record  can  most 
easily  be  kept,  updated,  and  transmitted  If  It  Is  In  machine  readable  form. 
This  data  base  can  be  used  In  conjunction  with  other  Information  such  as 
trouble  reports,  maintenance  records,  or  cost  data  to  determine  If  the  metric 
data  correlates  with  parallel  or  past  events.  If  an  environment  is  calibrated 
through  experience,  predictions  could  then  be  made.  All  of  these  capabilities 
can  be  realized  If  the  metric  data  Is  placed  In  a  data  base. 


An  Important  Ingredient  In  an  effective  software  quality  assurance  effort  is 
to  operate  In  such  a  fashion  that  measurement  does  not  interfere  significantly 
with  development,  test,  or  other  procedures  which  are  critical  in  production. 
If  metric  analysis  Is  done  by  machines  the  possibility  arises  to  move  a 
majority  of  the  quality  assurance  activity  off-line  of  the  development 
activity. 

For  these  three  reasons,  the  tedious  nature  of  examining  source,  the  need  for 
storage  and  retrlval  of  metric  data,  and  the  off-line  nature  of  SQA  activity, 
automation  of  metric  applications  Is  important  to  software  quality  metric 
Implementation. 

The  goals  of  this  project  were  to  make  this  process  more  timely,  reliable,  and 
economic.  Automating  the  collection  and  reporting  provides  these  three 
goals.  In  addition,  the  data  Is  maintained  in  a  data  base  for  use  as  a 
quality  assurance  management  information  system,  as  an  historical  record  of 
the  development  project,  and  as  a  repository  for  researchers  to  investigate 
combinations  of  data  items  to  form  new  metrics. 

The  prototype  version  of  the  AMT,  which  Is  described  in  more  detail  in  Section 
4  of  this  report  and  referenced  project  documents,  was  developed  to 
demonstrate  the  effectiveness  of  these  concepts.  25  module  level  measurements 
are  automatically  collected.  Thus,  of  the  16,433  measurements  that  could 
potentially  be  required  in  our  previous  example,  the  AMT  automates  collection 
of  2500.  The  AMT  data  base  accommodates  the  total  complement  of  data  items. 
The  measurements  not  taken  automatically  have  to  be  entered  into  the  data  base 
manually. 

The  25  data  items  collected  automatically  represent  27%  of  the  92  data  items 
measured  during  Implementation.  No  attempt  was  made  to  automatically  measure 
any  data  Items  related  to  requirements  and  design.  The  reasons  are  discussed 
In  the  next  paragraph. 


3-3 


3.3  OPERATIONAL  CONCEPT 

The  need  for  automated  support  of  the  application  of  the  metrics  is  tempered 
by  the  fact  that  the  metrics  we  have  defined  In  previously  funded  RADC  and 
USACSC  projects  not  only  relate  to  source  code  but  also  to  material, 
specifically  documentation,  normally  available  during  the  requirements  and 
design  phases  of  a  software  development.  Because,  at  the  present  time,  there 
are  few  formal  requirements  or  design  specification  languages  widely  accepted 
or  used  throughout  the  software  industry  and  the  material  produced  during 
these  phases  is  not  produced  in  machine  readable  form,  this  material  does  not 
lend  itself  to  automated  measurement.  However,  we  see  a  trend  toward  more  use 
of  formal  requirements  and  design  representations  such  as  SREM,  PSL/PSA,  PDL 
and  automated  analysis  tools  relating  to  them.  This  trend  will  allow 
expansion  of  the  AM T  in  terms  of  automated  collection  of  metric  information 
during  these  early  stages. 

The  AMT  is  a  system  that  is  used  during  all  phases  of  a  software  development 
process:  requirements  specification,  design  specification,  implementation, 

integration  and  test,  and  operation.  The  capabilities  provided  by  AMT  are: 
automatic  collection  of  specified  metric  data  from  machine-readable  materials, 
facilitation  of  collection  and  entry  of  other  metrics  data,  storage  and 
retrieval  of  metrics  data,  and  generation  of  different  reports  for  use  in 
tracking  software  quality.  These  capabilities  are  used  by  four  different 
types  of  users:  a  customer  or  user  of  the  software  system  being  developed, 
the  software  project's  quality  assurance  analyst,  the  software  development 
project  manager,  and  a  software  quality  researcher. 

With  these  constructs  in  mind,  the  automated  measurement  of  software  quality 
is  designed  to  work  in  the  following  way: 

(1)  At  the  beginning  of  a  project  the  quality  goals  of  the  project  are 
stated  and  desired  metric  values  are  determined. 

(2)  At  the  conclusion  of  the  requirements  phase,  worksheet  #1  is  manually 
completed  and  the  data  is  manually  entered  into  the  AMT's  data  base. 

(3)  When  the  preliminary  design  is  completed,  worksheet  #2a  is  manually 
completed  and  entered  into  the  data  base. 


(4)  As  various  detail  designs  are  completed,  worksheets  #2b  are  manually 
entered  for  each  module.  At  the  conclusion  of  the  design  phase  an 
update  to  #2a  Is  also  manually  placed  In  the  data  base.  Steps  2,  3, 
and  4  utilize  the  Requirements  Specification,  Preliminary  and 
Detailed  Design  documents,  test  plans,  preliminary  users  manual,  and 
other  material  normally  available  during  the  requirements  and  design 
phases  of  a  project. 

(5)  During  Implementation,  as  soon  as  COBOL  source  code  Is  placed  under 
configuration  management,  the  source  code  Is  measured  by  the  quality 
measurement  tool  and  this  data  Is  also  entered  Into  the  data  base. 
Additional  data.  Identified  on  worksheet  #3  Is  manually  entered. 
This  application  timing  Is  depicted  in  Figure  3.3-1. 

(6)  Various  updates  may  be  made  as  required. 

(7)  At  delivery  of  the  software  product.  It  is  anticipated  that  all 

metric  information  would  be  updated  to  reflect  the  current  state  of 
the  code  and  documentation. 

(8)  At  each  stage  of  application,  a  number  of  reports  would  be  generated 
to  provide  the  appropriate  Information  to  various  personnel  Involved 
In  the  development. 

One  primary  purpose  of  the  AMT  Is  to  calculate  the  metric  scores  from  the  data 
Input  from  the  worksheets.  These  scores  can  then  be  compared  with  desired 
scores.  This  operational  concept  is  depicted  In  Figure  3.3-2  and  Is 
compatible  with  the  methodology  described  In  the  Software  Quality  Measurement 
Manual  [MCCJ79],  Additional  helpful  Information  Is  available  to  support 

various  decisions,  quality  assurance  activities,  and  testing  activities. 

Figure  3.3-3  Identifies  the  support  provided.  This  support  Is  described 

further  In  Section  4  where  each  report  Is  defined. 

The  potential  of  the  software  metric  concepts  can  be  realized  by  their 
Inclusion  In  software  quality  assurance  programs.  Their  Impact  on  a  quality 
assurance  program  Is  to  provide  a  more  disciplined,  objective  approach  to 
quality  assurance  and  to  provide  a  mechanism  for  taking  a  life  cycle  viewpoint 
of  software  quality.  The  benefits  derived  from  their  application  are  realized 
In  life  cycle  cost  reduction. 
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Figure  3.3-1  Application  of  the  Metric  Worksheets 
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Figure  3.3-3  Support  to  Personnel 


The  measurement  concepts  complement  current  Quality  Assurance  and  testing 
practices.  They  are  not  a  replacement  for  any  current  techniques  utilized  in 
normal  quality  assurance  programs.  For  example,  a  major  objective  of  quality 
assurance  is  to  assure  conformance  with  user/customer  requirements.  The 
software  quality  metric  concepts  described  in  this  report  provide  a 
methodology  for  the  user /customer  to  specify  life-cycle-oriented  quality 
requirement  usually  not  considered,  and  a  mechanism  for  measuring  if  those 
reauirements  have  been  attained.  A  function  usually  performed  by  duality 
assurance  personnel  is  a  review/audit  of  software  products  produced  during  a 
software  development.  The  software  metrics  provide  formality  and 
quantification  to  these  reviews/audits  of  the  documents  and  code.  The  metric 
concepts  also  provide  a  vehicle  for  early  involvement  in  the  development  since 
there  are  metrics  which  apply  to  the  documents  produced  early  in  the 
development. 

Testing  is  usually  oriented  toward  correctness,  reliability,  and  performance 
(efficiency).  The  metrics  assist  in  the  evaluation  of  other  qualities  such  as 
maintainability,  portability,  and  flexibility. 

During  the  initial  design  phase  of  the  AMT  project,  an  informal  requirements 
definition  and  operations  concept  reflecting  the  discussions  in  this  paragraph 
were  documented  to  ensure  a  common  understanding  between  the  RADC  and  USACSC 
project  offices  and  the  development  team.  These  concepts  were  presented  at  a 
review  at  RADC.  The  informal  requirements  definition  and  operations  concept 
were  not  called  for  in  the  statement  of  work,  but  were  developed  to  supplement 
the  requirements  described  in  the  statement  of  work.  They  were  not  formally 
delivered. 

3.4  DESIGN  OF  AMT 

The  product  of  the  design  phase  of  the  AMT  project  was  a  Design  Plan,  CDRL 
A001.  The  Design  Plan  contained  a  detailed  design  of  the  AMT,  a  Data  Base 
Specification,  a  tool  survey  and  a  DBMS  survey,  the  implementation  schedule, 
and  the  plan  for  applying  metrics  to  the  AMT  development.  Each  of  these  steps 
in  the  design  of  the  AMT  will  be  described  in  this  section. 
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3.4.1  DESIGN  GOALS  OF  AMT 

There  were  specific  design  goals  identified  for  the  AMT  prototype.  A  highly 
usable  tool  that  could  eventually  operate  in  a  number  of  environments  in 
conjunction  with  other  software  and  project  management  tools  was  desired.  The 
software  quality  measurement  framework  was  used  to  identify  the  design  goals. 
Table  3.4. 1-1  provides  the  formal  definitions  of  the  quality  factors,  from 
which  three  were  chosen  to  be  emphasized  during  the  development  of  the  AMT. 

The  statement  of  work  identified  Portability,  Flexibility  and  Interoperability 
as  important  quality  goals  to  be  emphasized  in  the  AMT  development.  They  were 
identified  as  important  because  of  the  following  facts: 

(1)  Portability  was  considered  to  be  important  because  the  AMT  was 

developed  for  a  Honeywell  H6180/GC0S  computer  but  may  eventually  be 

transported  to  Honeywell  H6180/MULTICS,  IBM  370/0S,  and  PDP 

TAS 

11/70/  environments. 

(2)  Flexibility  was  an  important  design  consideration  because  the  AMT  is 
a  prototype  and  additional  requirements  will  be  forthcoming.  Also  the 
tool  will  eventually  be  used  in  a  variety  of  environments,  and  have 
to  process  other  languages  besides  COBOL. 

(3)  Interoperability  was  important  because  in  any  particular  environment 
where  the  AMT  might  be  used,  other  software  tools  might  be  available 
that  are  sources  of  metric  information.  We  would  want  to  be  able  to 
easily  interface  the  AMT  with  these  other  tools  to  take  advantage  of 
the  metric  information  the  other  tools  collect  automatically. 


[ij 


m 


LIFE 

CYCLE 

STAGES 


CORRECTNESS 


RELIABILITY 


INITIAL 

PROOUCT  EFFICIENCY 
OPERATION 

INTEGRITY 


USABILITY 


MAINTAINABILITY 


Extent  to  which  a  program  statisfies  its 
specifications  and  fulfills  the  user's 
mission  objectives. 

Extent  to  which  a  program  can  be  expected 
to  peform  its  intended  function  with 
required  precision. 

The  amount  of  computing  resources  and  code 
required  by  a  program  to  perform  a  function. 

Extent  to  which  access  to  software  or  data 
by  unauthorized  persons  can  be  controlled. 

Effort  required  to  learn,  operate,  prepare 
input  and  interpret  output  of  a  program. 


Effort  required  to  locate  and  fix  an  error 
in  an  operational  program. 


PRODUCT 

REVISION 


TESTABILITY 


FLEXIBILITY 


Effort  required  to  test  a  program  to  insure 
it  performs  its  intended  function. 

Effort  required  to  modify  an  operational 
program. 


Effort  required  to  transfer  a  program  from 
one  hardware  configuration  and/or  software 
system  environment  to  another. 

Extent  to  which  a  program  can  be  used  in 
other  applications  -  related  to  the 
packaging  and  scope  of  the  functions  that 
programs  perform. 


INTEROPERABILITY 


Effort  required  to  couple  one  system  with 
another. 


3.4.2  DESIGN  APPROACH 

Specific  design  approaches  were  taken  to  satisfy  these  quality  goals. 

In  part  the  modularity  of  the  system  design  enhances  the  qualities  of 
flexibility  and  portability.  The  AMT  was  divided  into  six  functional 
subsystems:  the  Executive  Services  (EXS),  the  Data  Base  Management  Services 
(DMS),  the  Automated  Measurement  Services  (AMS),  the  Preprocessing  Subsystem 
(PPS),  the  Report  Generation  Services  (RGS),  and  the  AMT  Utility  Services 
(UTL)  as  illustrated  in  the  AMT  hierarchy  diagram  shown  in  Figure  3.4-1. 

The  EXS  provides  the  interface  between  the  user  and  the  AMT.  The  EXS 
interprets  the  user's  commands  and  performs  all  of  the  necessary  calls  to  the 
other  AMT  functions  that  actually  carry  out  the  actions  requested  by  the 
user.  To  provide  greater  portability  the  AMT  has  its  own  Data  Base  Management 
Services  (DMS).  The  DMS  provides  the  capability  to  store  and  retrieve  metric 
data  from  a  random  access  file.  The  data  base  is  described  in  more  detail  in 
Section  4  and  in  the  Data  Base  Description  Document,  CDRL  A011.  The  primitive 
operating  system  dependent  functions  of:  opening  a  file,  closing  a  file, 
reading  a  record  from  the  file,  and  writing  a  record  into  the  file  are 
performed  by  the  AMT  Utility  Services  (UTL).  These  functions  were  isolated  to 
facilitate  modification  for  transporting  the  system  to  other  environments. 
The  Automated  Measurements  Services  (AMS)  extracts  certain  metric  Information 
directly  from  the  COBOL  source  code.  The  Preprocessing  Subsystem  (PPS)  is 
provided  soley  to  support  AMS  functions.  This  subsystem  has  no  direct 
interface  to  the  user  and  is  accessed  only  by  the  AMS.  The  basic  function  is 
a  generalized  parsing  system  to  take  source  code  input  by  an  AMT  user  and 
generate  parse  trees  representing  that  code. 

The  parse  tree  is  then  used  by  the  AMS  functions  to  produce  values  for  the 
metrics  worksheets.  It  should  be  noted  that  the  preprocessing  functions 
perform  their  own  data  management  services,  independent  of  the  DMS  functions. 
The  reason  for  this  separation  is  that  the  data  that  AMS  and  DMS  functions 
individually  manipulate  are  completely  different  in  form  and  content  and  the 
AMS  functions  are  the  only  functions  that  use  or  manipulate  that  particular 
type  of  data.  The  use  of  a  generalized  parsing  function  is  a  key  design 


COBOL 

AUTOMATED  MEASUREMENT 
TOOL 


3-13 


1075K 


choice.  By  describing  the  grammar  of  a  language  other  than  COBOL  and 
developing  a  scanner  for  that  language,  the  parser  will  produce  a  parse  tree 
representation  that  can  be  used  by  the  remainder  of  the  AMT  functions  with 
minor  modification.  Finally,  the  various  metric  reports  and  statistical 
analyses  are  performed  by  the  Report  Generation  Services  (RGS).  We  designed 
our  own  data  management  routines  primarily  to  enhance  the  portability  of  the 
system.  We  conducted  a  Oata  Base  management  System  survey  on  the  target 
environments  (HG180/GC0S,  HG180/Multics,  IBM  370/OS,  POP  11/70/IAS)  to 
evaluate  the  portability  issues.  Certainly  if  one  DBMS  had  existed  on  all 
four  environments  our  portability  problems  with  respect  to  data  would  have 
been  solved.  However,  this  was  not  the  case  and,  in  fact,  our  analysis 
determined  that  developing  our  own  data  management  routines  would  be  much  less 
expensive  than  having  to  convert  from  one  DBMS  to  another  when  we  wanted  to 
move  the  AMT  to  another  environment.  The  goals  of  the  Data  Management 
Services  were: 

•  The  data  base  must  be  portable  not  only  in  the  transfer  of  the  data 
base  functions  from  one  machine  to  another,  but  the  data  within  the 
data  base  needs  to  be  portable  for  research  reasons. 

•  The  data  must  be  easy  to  access  and  insert. 

•  The  user  must  be  able  to  enter  and  exit  the  data  base  with  ease. 

•  The  data  base  needs  to  be  highly  maintainable. 

•  The  data  base  needs  to  be  accessible  by  other  software  tools  in  a  given 
software  development  environment. 

t  No  on-line  access  via  a  general  query  language  was  provided. 

Appendix  C  contains  the  results  of  the  DBMS  survey. 
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Interoperability  was  designed  into  the  system  primarily  through  the  data  base 
design.  By  providing  routines  to  manipulate  the  data  with  respect  to  the  AMT 
data  base,  the  output  of  other  software  tools  could  be  accessed  and  inserted 
into  the  AMT  data  base.  Thus  if  the  AMT  was  being  used  in  an  environment 
where  PSL/PSA  was  utilized,  the  output  of  PSA  could  be  searched  by  a  software 
routine  and  appropriate  data  extracted  and  inserted  into  the  AMT  data  base. 
We  conducted  a  tool  survey  to  identify  potential  tools  for  interfacing  with 
the  AMT.  Several  potential  tools  were  identified.  The  results  are  in 
Appendix  D. 

During  the  design  phase,  an  implementation  schedule  was  developed.  The 
approach  taken  will  be  described  in  the  next  paragraph.  Also,  during  the 
design  phase,  we  began  applying  metrics  to  the  development  effort.  This 
application  was  an  experiment  to  assess  the  effectiveness  of  applying  metrics 
during  a  development.  The  approach  and  results  of  the  experiment  are  in 
Section  5  of  this  report. 

The  actual  design  of  the  AMT  was  conducted  using  structured  design 
techniques.  The  design  of  the  system  was  iteratively  decomposed  to  more 
detailed  descriptions  of  the  subsystems  shown  in  Figure  3.4-1  and  eventually 
to  detailed  designs  of  each  module.  Hierarchy  charts  for  each  subsystem  were 
generated,  HIPO  diagrams  were  constructed  for  each  module,  and  program  design 
language  descriptions  (PDL)  of  the  logic  were  prepared.  The  PDL  used  was  an 
Ada-like  language  developed  by  General  Electric.  Complexity  measures  were 
automatically  calculated  from  the  PDL' s.  These  metrics  were  utilized  during 
design  team  walkthroughs  to  evaluate  the  overall  complexity  of  the  design. 
The  design  was  documented  in  the  Design  Plan  (CDRL  A001)  and  in  part  in  the 
System/Subsystem  Description  Document  (CDRL  A009). 

3.4.3  USER  ORIENTED  CONCEPT 

Experience  with  software  quality  metrics  has  pointed  out  that  a  variety  of 
personnel  have  use  of  the  metric  data  for  a  variety  of  reasons.  Program 
Managers  are  interested  in  the  progress  of  a  project  with  regard  to  quality 
goals.  Developers  use  metric  data  to  aid  design  decisions  and  enforce 
standards.  Test  personnel  use  the  data  to  determine  test  strategies, 
emphasis,  and  level  of  effort. 
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•  i 


•  - 
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The  quality  assurance  staff  monitors  and  reports  on  violations  to  standards, 
goal  achievement,  and  quality  compliance.  Researchers  are  typically 
interested  in  historical  data. 


Given  these  variety  of  users,  ranging  from  sophisticated  to  uninitiated,  from 
experienced  to  untrained,  the  tool  was  designed  with  the  following 
charateri sties  in  mind: 

•  Transparent:  The  actual  operations  of  the  tool  should  not  interfere 
with  the  use  of  the  tool. 

•  System  independent:  The  user  should  have  to  use  a  minimum  of  system 
language  to  use  the  tool. 

•  Forgiving:  The  tool  should  have  the  capability  of  trapping  errors  and 
recovering  gracefully  to  minimize  the  impact  on  the  user. 

•  Helpful:  The  software  should  carry  as  much  interactive  training 

functions  as  possible  within  the  tool. 

•  User  flexible:  For  the  experienced  user  of  the  tool,  options  should  be 
available  to  increase  the  speed  with  which  the  user  can  interact  with 
the  system. 

•  In  general  the  tool  should  be  “user  friendly." 

A  User's  Manual  (CDRL  A012)  was  developed  to  provide  guidance  to  users  on  the 
use  of  the  AMT. 

3.5  IMPLEMENTATION  APPROACH 

The  implementation  of  the  AMT  was  done  in  IFTRAN2,  a  General  Research 
Corporation  structured  programming  preprocessor  to  FORTRAN.  IFTRAN2  provided 
a  structured  FORTRAN-based  language  for  developing  the  source  code.  As  a 
result,  the  code  is  well  structured  and  easier  to  read.  The  implementation 
was  conducted  incrementally  over  an  11  month  period.  The  initial  capability 
developed  was  the  Executive  Services  Subsystem.  This  subsystem  processes  the 
user  command  language.  By  providing  this  function  first,  future  users  could 
begin  training  and  gain  familiarity  with  the  user  interface.  Also  the 
Preprocessing  Subsystem  was  started  early  during  the  implementation  phase. 
The  parser  portion  of  this  subsystem  was  an  existing  system.  We  had  to 
describe  the  COBOL  grammar  in  a  Backus-Naur-Form  (BNF)  like  language,  however, 
and  because  we  could  find  no  such  description  of  COBOL  we  started  this  task 
early  to  avoid  unexpected  difficulties. 
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The  next  subsystem  to  be  developed  was  the  Data  Management  Services.  This 
allowed  demonstration  of  the  capability  to  manually  enter  and  extract  data 
from  the  data  base.  The  Utilities  Subsystem  was  also  developed  at  this  time  . 

The  Automated  Measurement  Subsystem  and  Report  Generation  Subsystem  were  then 
completed  to  provide  the  full  operational  capability. 

Documentation  of  the  Implementation  of  the  AMT  was  provided  in  the  Program 
Specification  (CDRL  A010),  the  Program  Maintenance  Manual  (CDRL  A013),  and  the 
source  code  provided. 

Because  of  the  inefficiencies  of  developing  the  software  remotely  on  the  RADC 
H6180,  the  AMT  subsystems  were  prototyped  on  a  General  Electric  VAX  11/780. 
To  enhance  the  portability  of  these  prototypes  to  the  RADC  H6180,  specific 
standards  and  conventions  were  developed  and  followed.  These  standards  and 
conventions  are  described  in  Appendix  B.  Final  system  integration  and 
modification  was  done  remotely  on  the  RADC  H6180. 

3.6  AMT  TEST 

A  Test  Plan  (CDRL  A015)  was  developed  and  submitted  for  review  by  RADC  and 
USACSC  at  the  end  of  the  design  phase.  This  test  plan  Incorporated  a  strategy 
of  testing  each  subsystem  Increment  as  it  was  developed,  integrating  them 
stepwise  Into  the  development  environment,  performing  system  test, 
transferring  to  the  RADC  target  environment,  and  performing  regression  system 
tests.  The  RADC  testing  was  done  remotely.  The  tests  were  based  primarily  on 
demonstrating  the  functional  capabilities  of  the  AMT  and  Its  error  handling 
capabilities,  using  five  COBOL  programs  provided  by  the  USACSC.  The  results 
of  the  tests  were  documented  In  a  Test  Analysis  Document  (CDRL  A014) 

3.7  AMT  TRAINING 

The  last  phase  of  the  project  was  to  develop  a  training  outline,  provide 
training  in  the  form  of  a  demonstration  at  RADC,  and  deliver  the  tool  and 
documentation,  including  the  analyses  performed.  The  training  material  (CDRL 
A006)  was  delivered  also.  The  AMT  User's  Manual  (CDRL  A012)  provides  examples 
from  operating  the  AMT. 
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SECTION  4 
AMT  DESCRIPTION 
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4.1  OVERVIEW 

The  AMT  is  a  system  to  be  used  in  support  of  the  quality  assurance  function 
during  software  development  and  maintenance.  Four  types  of  users  are 
envisioned  for  the  system,  a  customer  or  user  of  a  software  system  to  be 
developed,  a  quality  assurance  analyst,  the  software  development  project 
manager,  and  a  software  researcher.  The  objective  of  the  system  is  to  provide 
quality  assurance  information  to  these  users  in  the  form  of  software  quality 
metrics. 

The  system  provides  five  major  services  to  accomplish  its  function  of 
providing  software  quality  metrics  to  its  users. 


•  Automatic  Metric  Collection 

•  Manual  Metric  Collection 

•  Storage  and  Retrieval 

•  Report  Generation 

•  User  Messages 


In  order  to  support  these  services,  the  subsystem  design  shown  in  the 
functional  block  diagram  in  Figure  3.4. 1-1  was  developed.  The  major  functions 
are: 


•  Executive  Services  (EXS) 

•  Database  Management  Services  (DMS) 

•  Automatic  Measurement  Services  (AMS) 

•  Report  Generation  Services  (RGS) 


The  relationship  between  the  services  provided  by  the  AMT  and  the  functional 
areas  is  shown  In  Figure  4.1-1.  The  following  paragraphs  describe  each  of  the 
functional  areas.  Detailed  hierarchy  charts  of  each  subsystem  are  contained 
in  the  System/Subsystem  Specification  (CDRL  A009). 
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Figure  4.1-1  Services 
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4.2  EXECUTIVE  SERVICES 

The  function  of  the  Executive  Services  (EXS)  is  to  provide  the  interface 
between  the  user  and  the  AMT.  Thus,  EXS  interprets  the  user  commands  and 
performs  all  the  necessary  calls  to  other  AMT  functions  to  actually  carry  out 
the  actions  indicated  by  the  user  with  his  command.  The  EXS  monitors  system 
status  and  reports  all  errors  trapped  by  itself  or  any  other  subsystem.  Built 
in  debugging  functions  which  allow  a  user/programmer  to  trace  real-time 
program  execution  are  also  controlled  by  the  EXS.  The  EXS  queries  for 
complete  command  information  and  also  provides  user  'help'  information. 

4.2.1  COMMAND  LANGUAGE 

The  command  language  processed  by  EXS  is  as  follows: 

CR  Carriage  Return 

Responds  with  prompt. 

CREATE  databasename 

Creates  a  file  for  storing  worksheet  data.  Uses  system  file 
manipulation  routines. 

DELETE  Modulename 

This  command  deletes  a  module  from  the  current  database 
E  Exits  the  user  from  the  current  task. 

END  Terminates  AMT  session.  Closes  all  open  files  prior  to  termination. 

ENTER  modulename 

Used  to  identify  new  modules  for  which  data  is  to  be  stored  in 
current  data  base. 

GET  worksheet  number  [sectionnumber]  [itemnumber]  [modulename] 

Retrieves  items  currently  stored  in  data  base.  Retrieval  is  based  on 

individual  items,  sections  from  a  worksheet,  or  an  entire  worksheet. 
For  worksheets  that  are  at  module  level,  module  name  must  be 
specified. 
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HELP 


[commandname] 

Provides  text  which  explains  syntax  of  each  command  and  function. 
Without  command  name,  It  provides  list  of  available  commands. 


MEASURE  sourcefilename  modulename 

Causes  automated  source  code  metric  collection  to  be  initiated  using 
the  source  code  in  sourcefilenamefile.  Data  collected  is  stored  in 
appropriate  worksheet  for  module  name. 


PUT 


worksheetnumber  [sectionnumber]  [itemnumber]  [modulename] 

Allows  storage  of  values  in  the  database.  The  system  prompts 
operator  for  value  or  values  by  placing  worksheet  identifying  phrase 
or  question  on  terminal.  Prompts  are  for  individual  items,  or  for 
each  item  within  a  whole  worksheet.  For  worksheets  that  are 
organized  by  module,  the  modulename  must  be  entered  or  if  it  is  not 
entered,  a  prompt  requesting  it  is  displayed. 


REPORT 


reportname  [Printer] 

Generates  the  report  requested.  The  reports  are  presented  at  the 
terminal.  Certain  reports  require  further  input  and  the  operator  is 
prompted  for  further  input. 


SET 


databasename 

The  data  base  for  subsequent  commands  to  interact  with  is  identified 
by  this  command.  Only  one  database  may  be  processed  at  any  one 
time.  A  SET  command  supercedes  previous  SET'S.  The  data  base  had  to 
have  been  created  for  the  SET  to  work. 


[  ]  indicates  optional  data 

4.2.2  GENERAL  CONVENTIONS  FOR  ENTERING  A  COMMAND 

When  entering  commands  and  keywords  the  user  need  only  type  the  first  three 
characters  of  the  command  name,  e.g.,  CRE  for  CREATE,  the  one  exception  to 
the  rule  is  the  worksheet  numbers,  for  which  four  characters  are  required 
i.e.,  WS2A  and  WS2B  for  worksheet  2A  or  worksheet  2B. 


The  various  parts  of  a  command  must  be  separated  by  one  or  more  spaces  l.e. , 
CRE  PR0G1.  For  commands  that  have  parameters  the  user  may  type  just  the 
command  name  followed  by  a  carriage  return.  The  AMT  will  prompt  the  user  for 
the  necessasy  parameters.  For  example,  when  CRE  is  entered  AMT  prompts  with 
ENTER  DATABASE  NAME:  Each  command  or  command  part  must  be  terminated  with  a 
carriage  return. 

When  entering  responses  to  the  Yes/No  questions  a  "Y"  may  be  typed  for  "Yes", 
and  "N"  may  be  typed  for  a  "No".  Not  applicable  responses  must  be  indicated 
by  entering  "NA". 

4.3  DATA  BASE  MANAGEMENT  SERVICES 

The  Database  Management  Services  (DMS)  provide  for  the  storage  and  retrieval 
of  raw  metric  data.  The  ability  to  create,  open,  close,  and  expand  AMT  data 
bases  is  also  provided  by  this  subsystem.  While  individual  data  base  items 
can  be  referenced,  the  data  structure  used  most  often  by  DMS  is  the 
worksheet.  This  implementation  provides  rapid  and  efficient  access  to  the 
data  base  entries  and  their  values.  System  dependent  functions  such  as  file 
handling  and  disk  access  are  isolated  in  the  Utilities  Subsystem  (UTL). 

4.3.1  DATABASE  DESIGN 

The  worksheet  oriented  organization  of  the  metrics.  The  worksheets  are 
defined  in  the  Software  Measurement  Manual  [MCCJ19].  Samples  are  in  Appendix 
A. 

The  logical  structure  of  the  data  base  is  described  in  Figure  4.3-1.  A 
separate  data  base  is  maintained  for  each  system  entered  by  a  user  or  users  of 
the  AMT.  Logical  records  correspond  to  worksheets,  with  two  distinguished 
worksheets  (1  and  2a)  for  which  only  one  copy  each  exits.  This  Is  because 
these  worksheets  are  system  level  and  refer  to  all  modules.  Multiple  copies 
of  worksheets  2b  and  3  are  provided  since  these  correspond  to  module  level 
metrics.  Reference  is  made  by  worksheet  number,  section  number  and  item 
number  and,  in  the  case  of  worksheets  2b  and  3,  by  module  name. 


Individual  worksheet  data  items  occur  as  elements  of  an  array  associated  with 
each  worksheet/logical  record.  Logical  records  are  linked  by  number  to  a 
particular  module  name,  the  module  names  are  kept  in  a  list  which  is  physically 
stored  at  the  beginning  of  the  data  base  file.  The  prototype  version  of  the  g 

AMT  has  a  limitation  per  data  base  of  50  modules.  The  system  organization  of 
the  data  base  using  the  GCOS  file  management  system  is  shown  in  Figure  4.3-2. 

The  logical  record  number  of  a  particular  module's  worksheet  is  thus  calculated 

by  finding  the  corresponding  entry  number  of  the  module  in  the  module  name  list 

and  adding  an  offset  number  (which  is  implementation  dependent).  Particular 

data  items  can  then  be  obtained  by  using  the  array  index  associated  with  the 

item.  Array  indices  for  a  particular  data  item  in  a  worksheet  are  stored  in  a  v 

pointer  table  which  is  indexed  by  the  triple  (worksheet  number,  section  number, 

item  number).  This  table  is  pre-defined  and  is  stored  as  a  DATA  statement  (see 

Figure  4.3-3).  A  more  complete  description  of  the  data  base  implementation  is 

in  the  AMT  Data  Base  Description  document  (CDRL  A011).  m 

Raw  metric  data  is  stored  in  the  data  base  in  two  ways.  First,  it  can  be 
stored  manually  by  the  user.  The  user  can  enter  module  names  using  the  ENTER 
command.  This  command  enters  the  module  name  in  the  data  base  and  reserves  a  I 

designated  area  for  storing  worksheets  2b  and  3*  s  data  for  that  module.  The 
user  can  also  enter  metric  raw  data  from  one  of  the  worksheets  by  using  the  PUT 
command.  The  PUT  command  can  be  used  to  prompt  the  user  for  one  of  the  data 
items  in  a  section  or  a  worksheet  or  it  can  be  used  to  enter  one  specific  value  p 

individually. 

These  metric  values  are  calculated  each  time  a  user  tries  to  generate  reports. 

This  recalculation  of  the  metric  values  is  performed  to  insure  current  values. 

The  slight  processing  overhead  is  considered  worth  the  benefit  of  currency. 

The  metric  values  calculated  are  stored  in  local  arrays.  The  system  level 
metrics  are  In  single  dimension  arrays  while  the  module  level  metrics  are  in 
two  dimension  arrays.  This  physical  and  logical  structure  allows  for  the 
processing  flow  within  AMT  to  be  as  shown  In  Figure  4.3-4.  The  reason  raw 
metric  data  is  maintained  Instead  of  just  the  calculated  metric  values  Is  to 
allow  researchers  to  change  the  calculations  of  metrics  to  investigate  other 
algorithms. 
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Figure  4.3-1  Logical  Description  of  AMT  Data  Base 
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PHYSICAL  ORGANIZATION  OF  DATABASE 
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Figure  4.3-2  Physical  Organization  of  Data  3ase 
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4.3.2  FILE  SPECIFICATION  CONVENTIONS 

To  utilize  a  data  base  of  metric  information  within  the  AMT,  the  CREATE  and 
SET  commands  are  used.  When  using  these  commands  the  user  is  interfacing  with 
the  AMT  data  base.  The  AMT  takes  the  character  string  the  user  enters  as  part 
of  the  CREATE  of  SET  command  and  appends  .DAT  to  it.  For  example: 

CRE  TEST 

creates  a  data  base  file  called  TEST. OAT.  At  other  times,  the  user  may  want 
to  interface  with  a  file  other  than  an  AMT  data  base.  An  example  is  the 
MEASURE  command.  In  this  situation  the  AMT  accesses  a  file  which  has  source 
code  that  is  to  be  analyzed  by  the  AM T  parser  and  Automated  Measurement 
Subsystem.  That  file  was  established  using  the  system  editor  and  file 
management  system.  In  this  case,  the  user  must  specify  the  full  filename  of 
the  file  containing  the  source  code. 

For  example: 

MEASURE  PR061.CBL  MODI 

accesses  a  file  called  PR0G1.CBL  which  contains  the  COBOL  source  code  for 
MODI.  The  user  should  be  aware  of  the  file  specification/naming  conventions 
and  file  maintenance  procedures  of  the  computer  they  are  using  to  run  AMT  in 
order  to  name  and  maintain  the  AMT  files. 


4.4  AUTOMATED  MEASUREMENT  SERVICES 


The  amount  of  data  that  can  be  automatically  collected  is  limited  to  data  that 
can  be  derived  from  machine  readable  sources  such  as  design  materials 
generated  on  the  computer  and  the  actual  source  code.  This  paragraph 
describes  the  current  automatic  data  collection  capability  of  the  AMT. 


The  current  version  of  the  AMT  automatically  collects  and  stores  raw  data  from 
COBOL  source  code.  The  remainder  of  the  raw  data  required  to  calculate  the 
metrics  must  be  manually  collected. 


4.4.1  AUTOMATED  MEASUREMENT  OF  SOURCE  CODE 

Currently  the  AMT  collects  data  from  COBOL  source  code  for  individual 
modules.  A  total  of  25  different  measurements  are  collected  automatically. 
These  measurements  can  be  divided  into  the  following  broad  classes: 
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1.  Counts  of  total  number  of  code  statements  and  comment  statements; 

2.  Counts  for  individual  types  of  statements  e.g.,  input,  output,  exit, 
entry,  declarative,  etc,; 

3.  Counts  of  different  types  of  branching  statements  both  conditional 
and  unconditional;  and 

4.  Counts  of  operands  and  operators,  for  use  in  calculating  Halstead's 
measure. 

The  specific  data  items  automatically  collected  are  shown  in  Table  4.4. 1-1. 
Also  noted  in  the  table  is  the  entry  on  the  worksheet  that  this  item  relates 
to  and  the  metric  that  it  helps  calculate.  The  worksheet  entry  is  identified 
by  worksheet  number  (WS3),  section  number  (SI),  and  item  number  (II).  Thus  a 
notation  such  as  WS3,  SI,  II  is  read  as  worksheet  3,  section  I,  item  1. 

This  automated  support  accounts  for  25  of  the  92  worksheet  3  data  items,  or 
27%  of  the  measurements  required  at  the  implementation  phase  of  a 
development.  These  25  data  items  help  calculate  9  of  the  38  metrics  related 
to  implementation,  or  24%  of  those  metrics.  The  metric  calculation  is 
described  in  paragraph  4.5,  Report  Generation. 

The  automated  data  collection  is  performed  by  the  Automated  Measurement 
Services  (AMS)  and  the  Preprocessing  Services  (PPS)  subsystems.  The  user 
invokes  these  subsystems  using  the  MEASURE  command.  The  Preprocessing 
Services  uses  an  LL(1)  generalized  parser  to  decompose  the  COBOL  source  code 
contained  in  the  file  identified  by  the  MEASURE  command.  The  result  of  the 
parsing  is  a  parse  tree  representation  of  the  source  code.  A  description  of 
the  parser,  which  uses  a  Backus-Naur-Form  description  of  COBOL  grammar,  is  in 
the  AMT  System/Subsystem  Specification  document  (CDRL  A009). 

The  Automated  Measurement  Services  subsystem  traverses  the  parse  tree  and 
counts  the  various  data  items  and  enters  them  in  the  data  base.  More  detailed 
descriptions  of  the  design  of  these  subsystems  are  in  the  Design  Plan,  CDRL 
A001 . 
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The  significant  aspect  of  this  approach  is  that  other  language  grammers  can  be 
described  to  the  parser,  a  scanner  developed,  and  the  parser  will  produce  a 
parse  tree  representation  of  the  other  language.  With  careful  attention  paid 
to  the  token  representation,  the  AMS  will  be  able  to  process  this  parse  tree 
representation  of  the  other  language  with  little  or  no  modification. 

4.4.2  AUTOMATED  AID  TO  MANUAL  MEASUREMENT  OF  METRICS 

Paragraph  4.4.1  described  the  25  measurements  that  are  automatically  collected 
by  the  AMT.  There  are  31  measurements  during  requirements,  173  during  design, 
and  67  during  implementation  that  are  not  currently  automatically  collected. 
The  AMT  does  provide  some  automated  support  to  their  collection  and  storage  in 
a  data  base. 

The  AMT  will  automatically  generate  worksheet  forms  which  must  be  filled  out 
by  an  analyst/inspector.  These  worksheets  can  be  printed  with  any  current 
data  that  exists  in  the  data  base  displayed.  This  is  particularly  useful  for 
worksheet  3  which  will  be  partially  completed  automatically  by  the  AMT  when 
the  MEASURE  command  is  used. 

The  AMT  also  provides  the  PUT  command  that  facilitates  the  user  entering  data 
into  the  data  base.  The  PUT  command  is  described  in  paragraph  4.3  and  in  the 
AMT  Users  Manual  (CDRL  A012). 

The  standard  procedure  for  using  the  AMT  to  assist  in  manual  collection  of 
metric  data  is  as  follows: 

(1)  The  appropriate  worksheet  will  be  printed  at  the  terminal.  This 
worksheet  will  be  the  data  collection  form  for  the  inspector's  use. 

(2)  Reference  should  be  made  to  the  Metrics  Enhancement  Final  Report 
[MCCJ79VolI],  Software  Quality  Measurement  Manual  [MCCJ79Vol II],  and 
the  AMT  User's  Manual  [CDRL  A012],  These  references  provide  a 
description  of  the  metrics,  the  worksheets,  and  how  they  can  be  used 
in  context  of  the  AMT  respectively.  The  AMT  User's  Manual  provides  a 
copy  of  each  worksheet  (Appendix  C),  instructions  for  completing  the 
worksheets  (Appendix  D),  and  an  example  of  the  worksheets  completed 
for  a  COBOL  source  program. 


(3)  The  appropriate  source  material  should  be  gathered.  For  example,  the 
Requirements  Specification  Is  the  source  material  for  worksheet  1. 

(4)  The  source  material  should  be  briefly  read  for  both  format  and 
content. 

(5)  A  detailed  analysis  of  the  source  material  should  then  be  conducted 
using  the  questions  on  the  worksheet  as  directed  inspection  of  the 
source  material. 

(6)  Once  all  questions  are  answered,  then  the  inspector  should  document 
any  overall  observations  that  might  be  made  based  on  the  inspection. 

(7)  The  answers  to  the  worksheet  questions  should  then  be  entered  into 
the  AMT  data  base  using  the  PUT  command 

This  standard  procedure  will  become  routine  with  application  experience.  This 
is  especially  true  if  the  experience  is  gained  in  an  environment  where  the 
documents  prepared  follow  a  consistent  format  or  are  prepared  according  to  the 
same  military  standard.  In  these  cases,  the  information  sought  as  a  result 
of  the  worksheet  questions  typically  will  be  in  a  certain  section  of  the 
document. 

This  general  approach  provides  the  inspector  a  framework  in  which  to  inspect 
material,  specific  questions  to  answer,  and  a  directed  sequence  to  follow. 
This  consistency  and  quantification  in  the  inspection  process  enhances  the 
consistency  between  inspectors  and  makes  the  process  more  repeatable  and 
consistent  between  applications.  These  benefits  provide  better  inspection 
results,  more  feedback  to  the  developers  and  management,  and  therefore  aid  in 
achieving  a  higher  quality  software  product. 

4.4.3  USE  OF  OTHER  AUTOMATED  TOOLS 

The  AMT  was  developed  with  the  concept  of  eventually  interfacing  it  to  other 
software  tools  in  a  software  development  environment.  The  interfacing  would 
be  done  by  extracting  metric  data  available  from  the  processing  done  by  the 
other  tools  and  inserting  the  data  into  the  AMT  data  base  so  metrics  could  be 
calculated. 

A  program  would  have  to  be  written  which  extracts  the  appropriate  data  from 
the  output  file  of  a  tool  and  using  the  AMT  PUT  command,  inserts  it  into  the 
data  base.  Potential  tools  that  should  be  considered  are  Requirements 
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Specification  Language  processors/analyzers.  Program  Design  Language 
Processors/analyzers,  code  auditors,  code  instrumentors,  and  configuration 

management  tools. 

4.5  REPORT  GENERATION  SERVICES 

The  Report  Generation  Services  (RGS)  provides  the  user  the  ability  to  generate 
various  reports  that  reflect  the  contents  of  a  database.  Nine  reports  may  be 
requested  by  the  user  to  display  the  metric  data  in  a  variety  of  formats,  and 
by  performing  additional  calculations,  present  various  forms  of  data  both  at 
the  module  and  system  levels.  The  processing  that  is  done  is  shown  in  Figure 
4.3-4.  Basically,  data  is  extracted  from  the  data  base  to  calculate  metric 

values.  The  algorithms  for  performing  these  calculations  are  contained  in  the 

Report  Generation  Services  routines.  These  algorithms  are  defined  in  the 

Metrics  Enhancement  Final  Report  [MCCJ79]  and  in  the  Program  Specification 
Document  (CDRL  A010).  Samples  of  these  reports  are  included  in  the  Users 
Manual  and  in  Appendix  A.  Brief  descriptions  follow: 

4.5.1  MODULE  REPORT 

This  report  displays  the  catalog  of  modules  that  have  been  entered  into  the 
database.  It  provides  a  status  report  on  the  database. 

4.5.2  METRIC  REPORT 

This  report  calculates  the  value  of  each  metric  catagorized  by  factor  and  by 
development  phase.  This  report  is  used  to  determine  a  total  picture  of  the 
project  as  measurements  are  taken. 

4.5.3  EXCEPTION  REPORT 

The  exception  report  delivers  the  relationship  of  each  module  to  a  given 
threshold  value  of  a  particular  metric.  The  relationship  (less  than,  equal 
to,  or  greater  than)  and  the  threshold  value  is  input  from  the  user.  This 
report  can  be  used  to  identify  modules  whose  scores  do  not  meet  a  certain 
threshold,  identifying  them  as  potential  problems. 
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4.5.4  QUALITY  GROWTH  REPORT 

When  the  user  wishes  to  track  the  value  of  a  particular  metric  over  time,  the 
Quality  Growth  Report  will  furnish  a  tabular  display  of  the  scores  of  a 
selected  metric  over  the  phases  of  the  project.  This  report  is  used  to  track 
a  particular  metric  through  a  project  to  see  how  its  value  changes. 

4.5.5  NORMALIZATION  REPORT 

The  Normalization  Report  provides  the  user  with  the  overall  rating  of  a 
selected  quality  factor.  A  series  of  regression  equations  are  displayed  which 
have  been  empirically  derived  from  research.  The  current  metric  values  are 
substituted  in  the  equations  and  a  rating  for  the  selected  quality  factor  Is 
calculated.  Regression  equations  exist  for  the  quality  factors  Reliability, 
Maintainability,  Portability,  and  Flexibility  only.  The  normalization 
function  is  calculated  at  a  module  level. 

4.5.6  STATISTICS  REPORT 

The  Statistics  Report  provides  a  profile  of  COBOL  constructs  for  each  module. 

4.5.7  SUMMARY  REPORT 

The  summary  report  provides  a  summary  of  the  metric  scores  for  all  of  the 
modules  In  the  system. 

4.5.8  WORKSHEET  REPORT 

The  worksheet  report  displays  the  raw  data  entered  in  each  worksheet.  It 
represents  the  current  values  in  the  database.  It  is  used  to  verify  and  track 
data  entry. 

4.5.9  MATRIX  REPORT 

This  report  displays  the  average  and  standard  deviations  for  all  metrics 
values  for  all  modules.  This  report  displays  all  of  this  Information  In  a 
matrix  form  allowing  the  user  to  easily  identify  modules  with  metric  scores 
that  vary  from  the  system  average. 


"•  1 
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4.5.10  REPORT  SUMMARY 

The  reports  may  be  classified  as  to  their  primary  use: 

•  Descriptive 

•  Historical 

•  Diagnostic 

The  reports  that  are  descriptive  are  the  Summary,  Matrix,  Module,  and  Metric 
reports.  Their  common  characteristic  is  that  they  report  data  in  a  format 
implying  no  judgements  concerning  the  data.  The  Summary  Report  reports  all 
metric  scores  for  each  module  or  for  all  the  modules.  The  Matrix  Report 
displays  the  mean  and  standard  deviation  of  the  modules  for  each  metric.  It 
is  a  good  snapshot  of  the  data  in  the  data  base.  The  Module  Report  is  meant 
for  operational  personnel.  It  reports  those  names  of  the  modules  which  are  in 
the  data  base.  The  Metric  Report  is  a  more  detailed  output  which  displays  the 
metric  values  for  each  module  in  a  detailed  form. 

The  Historical  Reports  are  the  Quality  Growth  and  Worksheet  Reports.  The 
Quality  Growth  report  provides  the  quality  trend  of  a  module  through  the 
development  phases.  The  Worksheet  Report  gives  a  very  detailed  display  of  the 
raw  data  before  it  is  transformed  into  metric  scores.  It's  main  use  is  to 
track  data  entry  and  updates. 

The  Diagnostic  Reports  are  those  that  identify  potential  problem  areas.  They 
are  the  Normalization,  Exception,  and  Statistics  Reports.  The  Normalization 
Report  applies  the  regression  equations  derived  from  research  to  metric  values 
related  to  the  quality  factors  of  flexibility,  maintainability,  portability, 
and  reliability  at  the  module  level.  These  regression  equations  have  been 
developed  through  examination  of  previous  projects. 

Regression  equations  for  the  remaining  quality  factors  have  not  been 
established.  The  Exception  Report  provides  a  comparison  of  the  metric  scores 
with  predetermined,  user  supplied  values.  The  Statistics  Report  gives  a 
diagnostic  snapshot  of  any  module.  These  data  may  be  used  to  evaluate 
standards  or  identify  potential  problem  areas. 
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1 

The  typical  use  of  these  reports  is  described  in  Table  3.3-3.  The  table 
identifies  what  type  of  support  each  report  offers  different  job  functions. 

£  Additional  reports  can  be  added  with  relatively  minor  effort.  Reference  to 

the  report  would  have  to  be  added  into  the  Executive  Services  processing  and  a 

report  routine  written  using  the  GET  command  to  extract  appropriate  data  from 
the  data  base.  Reference  should  be  made  to  the  AMT  Maintenance  Manual  (CDRL 
|  A013)  and  the  AMT  Program  Specification  document  (CDRL  A010)  for  further 

insight  into  the  modifications  necessary  to  write  a  new  report. 


1 
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SECTION  5 

RESULTS  OF  QUALITY  METRIC  EXPERIMENT 


5.1  INTRODUCTION 

The  software  quality  metrics  concepts  were  applied  during  the  development  of 
the  Automated  Measurement  Tool.  This  is  the  first  formal  application  of  the 
software  metrics  defined  in  [MCCJ77A]  and  therefore  viewed  as  an  experimental 
demonstration  of  the  metric  concepts.  The  purposes  of  this  application  of  the 
metrics  were: 

(1)  Provide  additional  experience  with  applying  the  metrics  and 

validating  their  utility.  An  added  benefit  of  this  experience  is 

that  the  application  of  the  metrics  was  in-line  with  the  development 
effort  and  not  after  the  fact  as  past  applications  have  been. 

(2)  Provide  quality  assurance  feedback  to  the  development  team  and  the 

RADC  project  engineer.  The  application  of  the  metrics  was  planned  as 
a  complement  to  the  planned  testing  to  insure  production  of  an 

effective  software  product. 

(3)  Meet  the  quality  requirements  identified  in  the  statement  of  work. 
Some  specific  qualities  were  identified  as  being  important  to  the  AMT 
development  and  the  metrics  were  applied  to  provide  some  assurance 
that  emphasis  was  placed  on  their  inclusion. 

(4)  Provide  an  experimental  basis  for  generating  suggestions  on  how  to 
best  use  the  metrics  in  a  contractual  environment. 

This  section  describes  the  approach  to  performing  this  experimental 

application  of  the  concepts,  the  results  achieved,  and  lessons  learned. 

This  section  has  the  following  organization.  Paragraph  5.2  describes  the 
quality  goals  identified  for  the  contract.  Paragraph  5.3  describes  the 

process  followed  in  applying  the  worksheets  during  the  development. 

Paragraph  5.4  describes  the  results  of  the  application  of  the  metrics  in  terms 
of  scores  achieved,  observations  made  based  on  the  scores,  assessment  of  the 
metrics,  calculations  of  the  normalization  functions,  and  comparison  of  the 
scores  with  the  goals  established.  Paragraph  5.5  compares  the  metric  values 
achieved  during  the  AMT  development  with  those  observed  in  past  experiences. 
Paragraph  5.6  summarizes  the  lessons  learned  from  the  experiment. 
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5.2  QUALITY  GOALS  FOR  THE  AMT  DEVELOPMENT 


5.2.1  STATEMENT  OF  WORK  RELATED  QUALITY  GOALS 

A  high  level  statement  of  the  quality  goals  of  the  AMT  development  was 
mentioned  in  paragraph  4. 1.1. 3  of  the  statement  of  work. 

The  quality  factors  portability,  flexibility,  and  interoperability  were 
identified  as  important  to  the  AMT  product.  Portability  was  important  because 
the  system  design  must  consider  four  different  target  environments: 
H6000/GC0S,  H6000/MULTICS,  PDP  11/70  UNIX,  and  IBM  370/0S. 

Flexibility  was  important  because  a  subset  of  the  entire  set  of  metrics  will 
initially  be  automatically  collected.  The  system  may  be  extended  in  the 
future  to  collect  a  larger  subset.  Interoperability  was  important  because  a 
number  of  new  or  existing  software  tools  may  be  interfaced  with  the  system. 

5.2.2  SPECIFIC  QUALITY  GOALS  ESTABLISHED 

To  establish  more  specific  goals  against  which  to  measure  the  development 
effort,  the  Software  Quality  Requirements  Survey  Form,  shown  in  Table  5. 2. 2-1, 
was  completed  by  a  representation  of  RADC,  USACSC/AIRMICS,  and  the  development 
team.  A  form  of  the  Delphi  technique  was  used  in  that  each  individual 
completed  the  form  and  then  in  a  group  session  agreed  to  a  priorized  list  of 
the  quality  factors  important  to  the  AMT  development.  The  quality  goals 
decided  upon  are  identified  in  Table  5. 2. 2-2. 

The  first  six  factors:  Portability,  Flexibility,  Reusability, 
Interoperability,  Correctness,  and  Maintainability  were  especially  emphasized 
in  the  experiment  since  they  were  ranked  highest. 

Additionally,  specific  ratings  for  four  factors  and  for  several  metrics  were 
established.  The  ratings,  shown  in  Table  5. 2.2-3,  were  established  based  on 
previous  experience  and  are  set  at  the  specific  levels  indicated  as  part  of 
the  experiment.  The  previous  experience  refers  to  the  validation  efforts 
conducted  under  preciously  funded  research  efforts.  The  Software  Quality 
Measurement  Manual  [MCCJ79]  provides  additional  guidance  on  certain  metrics 
and  normalization  function  thresholds. 


5-2 


Table  5. 2. 2-1  Software  Quality  Requirements  Survey  Form 


The  11  quality  factors  listed  below  have  been  isolated  from  the  cur¬ 
rent  literature.  They  are  not  meant  to  be  exhaustive,  but  to  reflect 
what  is  currently  thought  to  be  important.  Please  indicate  whether 
you  consider  each  factor  to  be  Very  Important  (VI),  Important  (I), 
Somewhat  Important  (SI),  or  Not  Important  (NI)  as  design  goals  in  the 
system  you  are  currently  working  on. 


RESPONSE  FACTORS 
_  CORRECTNESS 


RELIABILITY 


EFFICIENCY 


INTEGRITY 


USABILITY 


MAINTAINABILITY 


TESTABILITY 


FLEXIBILITY 


PORTABILITY 


REUSABILITY 


INTEROPERABILITY 


DEFINITION 

Extent  to  which  a  program  satisfies  Its 
specifications  and  fulfills  the  user's 
mission  objectives. 

Extent  to  which  a  program  can  be  expected 
to  perform  Its  Intended  function  with 
fequlred  precision. 

The  amount  of  computing  resources  and  code 
required  by  a  program  to  perform  a  function. 

Extent  to  which  access  to  software  or  data 
by  unauthorized  persons  can  be  controlled. 

Effort  required  to  learn,  operate,  prepare 
Input,  and  Interpret  output  of  a  program. 

Effort  required  to  locate  and  fix  an  error 
In  an  operational  program. 

Effort  required  to  test  a  program  to  Insure 
It  performs  Its  Intended  function. 

Effort  required  to  modify  an  operational 
program. 

Effort  required  to  transfer  a  program  from 
one  hardware  configuration  and/or  software 
system  environment  to  another. 

Extent  to  which  a  program  can  be  used  In  other 
applications  -  related  to  the  packaging  and 
scope  of  the  functions  that  programs  perform. 

Effort  required  to  couple  one  system  with 
another. 


What  type(s)  of  application  are  you  currently  involved  in? 


Are  you  currently  in: 

_  1 .  Development  phase 

_  2.  Operations/Maintenance  phase 

Please  indicate  the  title  which  most  closely  describes  your  position: 

_  1 .  Program  Manager 

_ 2.  Technical  Consultant 

_  T.  Systems  Analyst 

_ h.  Other  (please  specify) _ 


12 
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Table  5. 2. 2-2  Quality  Requirements  for  AMT  (In  Order  of  Ranking) 


FACTOR 

CONSIDERATION 

PORTABILITY 

(VI) 

Targeted  for  IBM  370,  H6000,  PDP  11/70. 

FLEXIBILITY 

(VI) 

Tools  to  be  added  and  capabilities  enhance. 

REUSABILITY 

(VI) 

In  transporting  to  other  environments  and 
languages,  want  to  reuse  as  much  software  as 
possible. 

INTEROPERABILITY 

(I) 

Software  tools  to  be  interfaced  with. 

CORRECTNESS 

(I) 

Utility  of  AMT  depends  on  its  functioning 
correctly. 

MAINTAINABILITY 

(I) 

May  eventually  be  maintained  by  personnel 
other  than  developers. 

RELIABILITY 

(SI) 

Accuracy  of  metric  counts  and  quality  rating 
calculations  important. 

USABILITY 

(SI) 

To  be  used  by  managers  and  QA  analysts. 

TESTABILITY 

(SI) 

The  correctness  of  the  metric  data  collection 
must  be  demonstrated. 

INTEGRITY 

(NI) 

The  security  of  the  data  base  is  not  really 
critical. 

EFFICIENCY 

(NI) 

Processing  efficiency  not  critical. 

Where 

VI  is  Very  Important 

I  i s  Important 

SI  is  Somewhat  Important 

NI  is  not  important 
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Table  5. 2. 2-3  Specific  Quality  Goals 


FACTOR  RATINGS 

AMT 

c2 

MIS 

Flexibility 

.7 

.4 

.4 

Portability 

.75 

NM 

.6 

Maintainability 

.7 

.33 

.20 

Reliability 

.9 

.98 

.92 

METRIC  THRESHOLD  VALUES 

MO. 2  Modular  Implementation  Measure  .7 

GE.2  Generality  Checklist  .35 

SD.l  Quantity  of  Comments  .2 

SD.2  Effectiveness  of  Comments  .40 

50. 3  Descriptiveness  of  Implementation  Language  .50 

MI. 1  Machine  Independence  Measure  .2 

CS.l  Procedure  Consistency  Checklist  .6 

SI.l  Design  Structure  Measure  .75 

51. 3  Complexity  Measure  .23 

51. 4  Code  Simplicity  Measure  .50 

C0.1  Conciseness  Measure  .90 


NM  -  Not  Measured 
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The  values  selected  for  Flexibility  (.7)  and  Portability  (.75)  are  above 

Industry  average  since  they  were  Identified  as  very  Important  for  the  AMT. 

The  value  for  Maintainability  (.7)  was  selected  at  what  Is  considered  the 

Industry  average  because  It  was  Identified  as  Important.  The  value  for 

Reliability  (.9)  was  Identified  as  below  the  Industry  average  since  It  was 

identified  as  only  somewhat  important  to  the  AMT.  The  values  we  experienced 

on  previous  studies  are  also  indicated  in  Table  5. 2. 2-3.  These  values  were 

2 

measured  in  a  Command  and  Control  (C  )  environment  and  a  Management 

2 

Information  System  (MIS)  environment.  Because  of  the  nature  of  the  C 
application  and  the  fact  that  the  MIS  system  was  a  production  system,  the 

values  for  Reliability  were  higher  for  those  systems  than  for  the  AMT. 

The  Metric  values  identified  likewise  were  drawn  from  previous  experience  and 
depending  on  whether  the  quality  factor  they  related  to  was  considered 

important  or  not  to  the  AMT,  their  values  were  set.  The  AMT  scores  as  well  as 
a  discussion  of  the  threshold  values  set  and  how  they  compare  to  previous 

experience  is  in  paragraph  5.5. 

5.2.3  APPLICATION  METHODOLOGY 

The  procedures  we  used  to  apply  the  measurements  to  the  AMT  development  are 
essentially  those  described  in  the  Software  Quality  Measurement  Manual 
[MCCJ78].  Those  applied  are  briefly  highlighted  here: 

(1)  Established  Quality  Goals  (see  paragraph  5.2.2) 

(2)  Applied  Worksheet  1  to  the  informal  Requirements  Specifications  that 
were  generated  at  the  outset  of  the  project. 

(3)  Applied  Worksheet  2a  to  the  Design  Document  at  the  system  level. 

(4)  Applied  Worksheet  2b  to  the  Design  Document  at  the  module  level. 

This  application  was  made  at  the  beginning  of  the  implementation  of 

each  increment  because  it  was  at  that  point  that  the  detailed  designs 

of  the  modules  in  that  increment  were  set. 
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(5)  Applied  Worksheet  3  to  the  code. 

(6)  Metric  scores  were  calculated  from  the  worksheets. 

(7)  Observations  or  analyses  based  on  the  worksheet  data  and  the  metric 
scores  were  documented  (see  paragraph  5.4). 

(8)  Where  normalization  functions  existed,  they  were  calculated. 

(9)  The  worksheet  data,  metric  scores,  documented  observations  and 
quality  ratings  (calculated  normalization  function)  are  presented  in 
this  report. 

(10)  Where  possible,  automated  tools  were  utilized  to  apply  the  metrics. 
Tools  considered  for  use  are  Identified  In  Appendix  D. 

5.2.4  DESIGN  AND  IMPLEMENTATION  GUIDELINES 

Based  on  the  identification  of  certain  quality  factors  as  goals  for  the 
development,  some  specific  practices  were  Identified  for  the  development  team 
to  follow.  These  guidelines  are  Identified  In  Appendix  B.  These  guidelines 
were  derived  from  experiences  in  transferring  PDP  FORTRAN  code  to  H6000 
FORTRAN  code,  and  from  transferring  code  from  the  H6000  to  an  IBM  370. 

5.3  APPLICATION  OF  WORKSHEETS 

The  worksheets  that  are  part  of  the  Software  Quality  Measurement  Manual 
[MCCJ79]  were  the  major  vehicle  for  applying  the  measurements  during  the  AMT 
development.  Figure  3.3-1  illustrates  the  timing  of  their  application.  Note 
that  worksheet  1  was  applied  to  the  draft  Requirements  Specification  that  was 
written  In  December  1979.  That  specification  was  not  a  formal  deliverable  of 
the  contract.  Worksheet  2a  was  applied  to  the  Design  Plan  (CDRL  A001). 

Worksheet  2b  was  applied  to  the  HIPO  diagrams  and  Program  Design  Language 
description  of  each  module  in  the  Design  Plan.  The  worksheets  were  applied  at 
the  Initial  phases  of  each  subsystem  as  it  was  being  developed.  This  timing 
was  chosen  because  at  that  time  the  POL's  and  data  definitions  were  refined 
and  were  the  driving  documents  of  the  Implementation.  It  was  at  that  time 
that  Identifying  quality  problems  had  the  most  positive  Impact. 
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Worksheet  3  was  applied  at  the  latter  stages  of  each  incremental  development 
phase,  when  the  source  code  was  complete.  Had  an  automated  tool  been 
available  the  measurements  would  have  been  taken  several  times  during  the 
implementation. 

Worksheet  1  and  2a,  as  completed,  are  in  Figures  A-l  and  A-2  and  an  example 
worksheet  2b  and  3  are  in  Figure  A-3  and  A-4  in  Appendix  A.  These  latter  two 
worksheets  were  completed  at  a  module-level. 

5.4  RESULTS 

The  results  of  the  application  of  the  worksheets  were  reported  to  RADC  at  two 
times  during  the  project.  The  first  time  was  during  a  review  of  the  Design 
Plan  at  RADC.  The  second  time  was  at  the  completion  of  the  delivery  of  the 
AMT.  The  results  were  used  by  the  development  team  thrughout  the  development 
effort  to  assess  the  quality  of  their  design  and  Implementation. 

The  results  of  the  application  of  the  worksheets  were  analyzed  at  three 
levels.  First,  some  general  observations  were  made  based  on  the  application 
of  the  worksheets.  Second,  the  metric  scores  were  calculated  and  reported. 
Third,  the  metric  scores  were  compared  with  the  quality  goals  for  the  project. 

5.4.1  REQUIREMENTS  AND  DESIGN 

5. 4. 1.1  General  Observations  At  Requirements  And  Preliminary  Design 

Table  5. 4. 1.1-1  contains  the  general  observations  made  based  on  the 
application  of  Worksheets  1  and  2a.  These  were  applied  to  the  draft 
requirements  document  prepared  In  the  first  month  of  the  the  contract  and  the 
design  document,  which  was  the  product  of  the  first  phase  of  the  contract  and 
represents  a  system-level  view.  These  worksheets  and  observations  were  made 
during  the  design  phase  of  the  contract  and  reported  to  the  RADC  and 
USACSC/AIRMICS  project  engineers  at  the  design  review. 

The  worksheet  applications  revealed  that  requirements  dealing  with  such 
attributes  as  security,  error  tolerance,  performance,  and  Interfacing  with 
other  systems,  had  not  been  specified.  Since  security  and  performance  were 


-• 
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Table  5. 4. 1.1-1  Observations  Based  on  Worksheet  Inspection  of  Requirements 

Specification  and  Preliminary  Design  Specification  during 
Design  Phase  of  the  Project 


REQUIREMENTS  (Worksheet  1) 

-  No  flow  of  processing  and  decisions  during  that  flow  described 
Operations  concept  did  not  really  describe  scenario  of  use 

-  No  reliability  requirements  specified;  error  tolerance,  error  recovery 

-  No  access  controls  required 

-  No  discussion  of  user  interface  except  for  command  language 

-  No  performance  requirements  stated 

-  Provisions  for  interfacing  with  other  systems  lacking 

PRELIMINARY  DESIGN  SPEC  (Worksheet  2a) 

-  No  error  reporting/control  system  in  effect 

-  Error  conditions  not  identified  yet 

-  No  called/call  matrix  for  modules  yet 

-  No  estimates  on  run  times  or  storage  requirements  yet 

-  No  access  controls  provided 

-  Other  tools  to  interface  with  have  not  been  identified 

-  User  Manual  not  written  yet  (outline  has  been) 

-  No  Test  Plan  yet 
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not  major  quality  goals  these  were  not  considered  for  correction.  Since 
Interoperability  was  important  to  the  AMT,  corrective  action  in  the  form  of  an 
analysis  of  how  the  AMT  would  interface  with  other  tools  was  conducted.  The 
results  of  the  analysis  were  built  into  the  design  of  the  Data  Management 
Subsystem. 

The  worksheet  application  to  the  preliminary  design  material  revealed  that  a 
call /cal  led  matrix  had  not  been  generated  at  the  subsystem  level  and  that 
error  handling  in  the  system  had  not  been  described.  8oth  of  these 
deficiencies  were  corrected  by  the  time  the  Design  Plan  (CDRL  A001)  was 
delivered.  An  update  to  worksheet  2a  at  detailed  design  was  not  done  but  one 
was  done  at  the  end  of  the  contract.  This  update  included  the  metrics  related 
to  the  Test  Plan  and  User's  Manual.  All  worksheets  have  been  delivered  to 
RADC  as  part  of  the  AMT  data  base  and  as  a  separate  document. 

5. 4. 1.2  Metric  Scores 

Table  5.4. 1.2-1  contains  the  system  metric  scores  calculated  from  the 
application  of  Worksheet  1  and  2a.  Paragraph  5. 4. 1.3  contains  an  analysis  of 
these  scores.  Only  those  metrics  identified  in  paragraph  5.2  were  measured. 

5. 4. 1.3  Comparison  with  Quality  Goals 

The  factors  identified  as  very  important  and  important  were: 

PORTABILITY: 

The  only  indicator  of  portability  at  preliminary  design  time  is  the 
Modular  Implementation  measure  (score  of  .57)  which  is  average  based  on 
past  experience.  Measures  of  the  machine  independence  and  system 
independence  are  not  made  until  detailed  design.  At  the  end  of  the 
preliminary  design  phase  of  the  project  the  design  was  still  machine  and 
operating  system  independent. 

FLEXIBILITY: 

The  Modular  Implementation  measure  (.57)  and  the  Generality  of  the 
design  approach  (.43)  effect  the  flexibility  of  the  code.  These  scores 
represent  a  slightly  better  than  average  score  for  flexibility  according 
to  past  systems  we  had  measured.  While  these  are  system  level  metrics 
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Table  5. 4. 1.2-1  Metric  Scores  from  Initial  Application 
of  Worksheet  1  and  2a 


Completeness  (CP. 1 )  ,8 
Accuracy  (AY.l)  *  0 
Error  Tolerance-Input  Data  (ET.2)  0 
Error  Tolerance-Computational  Failures  (ET.3)  0 
Error  Tolerance-Hardware  Faults  (ET.4)  0 
Error  Tolerance-Device  Errors  (ET.5  0 
Access  Control  (AC.l)  ,67 
Access  Audit  (AA.l)  *  q 
Operability  (0P.1)  0 
User  Input  Interface  ( CM. 1 )  1 
User  Output  Interface  (CM. 2)  1 
Communications  Commonality  (CC.l)  1 
Data  Commonality  (DC.l)  1 


PRELIMINARY  DESIGN 


Traceability  (TR.l)  \ 

Completeness  (CP. 1 )  ,8 

Accuracy  (AY.l)  0 

Error  Tolerance-Control  (ET,1)  0 

Error  Tolerance-Hardware  Faults  (ET.4)  0 

Error  Tolerance-Device  Errors  (ET.5  0 

Design  Structure  (SI. 1 )  ,33 

Modular  Implementation  (MO. 2)  .57 

Generality  (GE.l)  .43 

Module  Testing  (IN.l) 

Integration  Testing  (IN. 2)  Test  Plan  not 

System  Testing  (IN. 3)  completed  yet 

Iterative  Processing  Efficiency  (EE. 2)  0 

Data  Usage  Efficiency  (EE. 3)  1 

Storage  Efficiency  (SE.l)  0 

Access  Control  (AC.l)  1 

Access  Audit  (AA.l)  0 

Operability  (OP. 1 ) 

Training  (TN.l)  User  manual 

User  Input  Interface  (CM.l)  not  written 

User  Output  Interface  (CM. 2)  yet 

Communcation  Commonality  (CC.l)  .5 

Oata  Commonality  (DC.l)  1 
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if  we  substitute  them  into  normalization  equations  we  get  a  rating  of 
approximately  .25  or  4  man-days  to  make  a  modification  to  the  system.  This  Is 
better  than  the  specified  goal  of  6  man-days  to  make  a  modification  (rating  * 
.7). 

REUSABILITY: 

At  the  preliminary  design  phase  of  the  development  the  indicators 
available  for  reusability  were  the  same  as  for  flexibility. 
INTEROPERABILITY: 

The  measures  related  to  Interoperability  are  the  Communication 
Commonality  measure  (score  of  1)  and  Data  Commonality  measure  (score  of 
1)  during  requirements  and  the  same  two  (score  of  .5  and  1 
respectively)  plus  Modular  Implementation  (.57)  during  preliminary 
design.  The  scores  are  high  in  this  case  because  we  had  recognized  the 
requirements  to  build  a  system  with  which  it  will  be  easy  to 
interface.  The  primary  interface  will  be  with  the  data  base.  To 
Interface  a  tool  with  AMT,  one  must  write  a  translation  or  interface 
routine  which  takes  the  output  of  the  tool  and  transforms  it  Into  the 
format  of  the  AMT  data  base.  The  AMT  data  management  routines  would  be 
available  to  facilitate  that  process. 

CORRECTNESS: 

The  two  measures  which  relate  to  completeness  at  requirements  and 
preliminary  design  are  the  Completeness  measure  (.8)  and  the 

Traceability  measure  (1).  The  Consistency  measures  (1)  were  high 
because  the  design  team  agreed  to  a  standard  design  notation. 
MAINTENANCE: 

The  Design  Structure  measure  (score  of  .33),  the  Modular  Implementation 
measure  (score  of  .57),  and  the  consistency  measures  (1),  relate  to 
maintainability.  The  Design  Structure  measure  was  slightly  lower  than 
past  experience  has  indicated  It  should  be  so  we  looked  at  it  In  some 
detail.  Several  modules  were  being  called  by  many  other  modules  at 
different  levels  of  the  system  hierarchy.  This  lowered  the  design 
structure  metric  score.  By  Identifying  these  modules  as  utilities  the 
complexity  of  the  design  was  decreased. 


5.4.2  DETAILED  DESIGN  AND  IMPLEMENTATION 


5.4.2. 1  Get  yal  Observations  At  Oetailed  Design  and  Implementation 

The  difficulty  in  developing  software  remotely  in  terms  of  turnaround  and 
obtaining  complete  output,  and  the  fact  that  we  had  to  apply  the  metrics 
manually  to  the  AMT  development,  prevented  as  effective  use  of  the  metrics  as 
would  have  been  desired.  Timeliness  is  especially  critical  during  detailed 
design  and  implementation  because  in  order  to  affect  the  design  and 
implementation  strategies  the  measurements  have  to  be  available  on  practically 
a  daily  basis.  This  was  not  possible.  The  metrics  were  applied  once  during 
the  detailed  design  (worksheet  2b)  and  once  when  the  code  was  complete 
(worksheet  3)  and  are  reported  here  to  provide  assurance  that  a  high  quality 
product  was  provided.  The  information  Is  also  valuable  for  future  extensions, 
modifications,  and  transporting  of  the  AMT. 

5.4. 2.2  Metrics  Scores 

Table  5. 4. 2. 2-1  provides  a  summary  of  the  metric  scores  calculated  from 
worksheets  2b  and  3  applied  to  each  module  In  the  system.  The  scores  shown 
are  averaged  over  each  subsystem  and  over  the  entire  system.  The  system 
average  score  is  calculated  by  taking  the  sum  of  each  subsystem  average  score 
multiplied  by  the  number  of  routines  measured  in  that  subsystem  and  dividing 
this  sum  by  the  total  number  of  routines  in  the  system.  The  measurements  are 
taken  from  58  modules  representing  over  12,000  lines  of  code.  The  breakdown 
of  modules  by  subsystem  is: 


Executive  Services  Subsystems  (EXS)  11 
Automated  Measurement  Subsystem  (AMS)  4 
Data  Management  Subsystem  (DMS)  12 
Utilities  Subsystem  (UTL)  7 
Report  Generation  Subsystem  (RGS)  24 


Not  Included  in  the  measurements  is  the  Preprocessing  Subsystem  (PPS)  which 
includes  the  parser.  This  was  existing  code  and  the  metrics  were  not  applied 
to  it  during  the  AMT  development. 

Individual  module  scores  were  available  through  the  Metrics  Report  and  also 
through  the  Matrix  Report.  Analysis  of  these  scores  are  In  the  following 
paragraph. 
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Table  5. 4. 2. 2-1 
Implementation  Metric  Scores 


METRIC 

SUBSYSTEM  AVERAGE  SCORES 

SYSTEM 

AVERAGE 

AMS 

EXS 

UTL 

*  DMS 

RGS 

ACCURACY 

AY.  1 

0 

.27 

.14 

.17 

.46 

.29 

CONCISENESS 

CO.  1 

1. 

1. 

1. 

1. 

.99 

.996 

COMPLETENESS 

CP.l 

.25 

.56 

.38 

.38 

.75 

.25 

PROCEDURE  CONSISTENCY 

CS.l 

.5 

.73 

.57 

.58 

.96 

.76 

DATA  CONSISTENCY 

CS.2 

.25 

.7 

.29 

.29 

.48 

.39 

ITERATIVE  PROCESSING  EFFICIENCY 

EE. 2 

.5 

.55 

.14 

.92 

.42 

.51 

DATA  USAGE  EFFICIENCY 

EE. 3 

.46 

.65 

.45 

.55 

.67 

.59 

ERROR  TOLERANCE  CONTROL 

ET.1 

.75 

.54 

.43 

.83 

.96 

.77 

ERROR  TOLERANCE  INPUT  DATA 

ET.2 

0 

.38 

.19 

.17 

.29 

.25 

ERROR  TOLERANCE  COMPUTATION 

ET.3 

.08 

.15 

.07 

.12 

.01 

.07 

OATA  STORAGE  EXPANDABILITY 

EX.  1 

0 

0 

.14 

.08 

.21 

.12 

COMPUTATIONAL  EXTENSIBILITY 

EX. 2 

0 

.05 

.07 

.08 

.04 

.05 

IMPLEMENTATION  GENERALITY 

GE.2 

.63 

.73 

.54 

.83 

.77 

.74 

MACHINE  INDEPENDENCE 

MI .  1 

.74 

.88 

.63 

.84 

.89 

.84 

MOOULAR  IMPLEMENTATION 

MO. 2 

.31 

.46 

.38 

.34 

.36 

.37 

QUALITY  OF  COMMENTS 

SO.l 

.43 

.37 

.71 

.59 

.36 

.46 

EFFECTIVENESS  OF  COMMENTS 

SD.2 

.45 

.60 

.44 

.58 

.63 

.58 

DESCRIPTIVE NESS  OF  LANGUAGE 

SD.3 

.75 

.91 

.71 

.92 

.96 

.9 

DESIGN  STRUCTURE 

SI.1 

.45 

.63 

.45 

.63 

.63 

.59 

COMPLEXITY 

SI. 3 

.09 

.12 

.02 

.09 

.17 

.12 

CODE  SIMPLICITY 

SI. 4 

.49 

.62 

.43 

.69 

.61 

.6 

SYSTEM  SOFTWARE  INDEPENDENCE 

SS.l 

.25 

.36 

.33 

.29 

.48 

.38 

TRACEA8ILITY 

TR.l 

0 

.18 

.14 

0 

.04 

.07 

i'll  I 


5.4. 2.3  Comparison  With  Quality  Goals 

Table  5.4. 2. 3-1  compares  the  subsystem  and  system  average  metric  scores  with 
the  metric  scores  identified  in  Table  5. 2. 2-3.  These  metric  scores  were 


Identified  at  the  beginning  of  the  project  as  goals  the  development  team  would 
attempt  to  meet.  The  values  chosen,  threshold  values,  were  chosen  because 
they  represent  either  average  or  above  average  scores  for  those  metrics  based 
on  past  experience.  They  were  not  contractual  requirements  but  were  set  as 
quality  goals  against  which  to  assess  the  software  development. 
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In  most  cases,  the  threshold  values  were  met  or  exceeded  providing  some 
confidence  in  the  quality  of  the  software  product.  There  were  a  few 
exceptions.  The  modular  implementation  metric  (MO. 2)  is  currently  measured 
differently  than  previously  specified.  The  modular  implementation  measure 
(MO. 2)  during  past  studies  included  the  following  measurements: 


•  Module  size  in  lines  of  source  code  (1  if  less  than  100,  0  if  greater  * 

than  100) 

•  Number  of  parameters  which  are  control  variables  divided  by  number  of 
total  calling  parameters. 

•  Input  data  controlled  by  calling  module  (1  if  yes,  0  if  no)  • 

•  Output  data  controlled  by  calling  module  (1  if  yes,  0  if  no) 

•  Control  returned  to  calling  modulue  (1  if  yes,  0  if  no) 

•  Is  temporary  storage  shared  by  call/called  modules  (1  if  no,  0  if  yes) 

» 

These  measurements  were  added  together  and  divided  by  six  to  get  the  metric 
value.  Two  additional  measurements  were  added  to  the  MO. 2  metric  in  the  AMT 
implementation.  These  were: 


t  1  divided  by  number  of  elements  passed  as  parameters  that  were  not 
variables 

•  1  divided  by  number  of  parameters  not  defined 


The  new  metric  value  Is  the  sum  of  the  above  six  elements  plus  the  new  two 
measurements  divided  by  eight.  However,  in  taking  these  latter  two 
measurements,  the  code  Inspectors  interpreted  both  as  zero  when  all  parameters 
passed  in  a  call  statement  were  variables  and  all  parameters  were  defined. 
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Table  5. 4. 2. 3-1  Comparison  of  Metric  Scores  with  Specified  Thresholds 


METRICS 


SUBSYSTEM  AVERAGE  SCORES 

SYS 

SPECIFIED 

AMS  EXS  UTL  DMS  RGS 

AVG 

THRESHOLD 

MODULAR  IMPLEMENTATION 

MO. 2 

.31 

.46 

.38 

.34 

.36 

.37 

.70 

GENERAL  CHECK  LIST 

GE.2 

.63 

.73 

.54 

.83 

.77 

.74 

.35 

QUANTITY  OF  COMMENTS 

SD.l 

.43 

.37 

.71 

.59 

.36 

.46 

.20 

EFFECTIVENESS  OF  COMMENTS 

SD.2 

.45 

.60 

.44 

.58 

.63 

.58 

.40 

DESCRIPTIVENESS  OF  LANGUAGE 

SD.3 

.75 

.91 

.71 

.92 

.96 

.90 

.50 

MACHINE  INDEPENDENCE 

MI.  1 

.74 

.88 

.63 

.84 

.89 

.84 

.20 

PROCEDURE  CONSISTENCY 

CS.l 

.5 

.73 

.57 

.58 

.96 

.76 

.60 

DESIGN  STRUCTURE 

SI.l 

.45 

.63 

.45 

.63 

.63 

.59 

.75 

COMPLEXITY 

SI. 3 

.09 

.12 

.02 

.09 

.17 

.12 

.23 

CODE  SIMPLICITY 

SI. 4 

.49 

.62 

.43 

.69 

.61 

.60 

.50 

CONCISENESS 

CO.l 

.92 

.94 

.98 

.91 

.99 

.94 

.90 

*These  values  were  computed  manually 
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This  misinterpretation  incorrectly  lowered  the  MO. 2  metric  value  by  two 
eighths  (2/8)  or  .25.  Thus  the  MO. 2  scores  should  have  been  .56,  .71,  .63, 
.59,  .61  for  the  subsystems  in  Table  5. 4. 2. 3-1  respectively  and  .62  for  the 
system  average.  The  scores  still  do  not  meet  the  threshold  but  average  89%  of 
the  threshold  score. 

The  complexity  measure  did  not  meet  the  goal  of  .23  set.  The  logic  of  some  of 
the  routines  to  calculate  the  metric  scores  and  to  parse  and  identify  the 
various  constructs  of  COBOL  is  quite  complicated.  The  average  achieved,  .12 
is  better  that  that  recommended  by  McCabe  [McCT]  which  equates  to  .1  in  our 
metric.  The  design  structure  metric  (SI . 1 )  was  slightly  lower,  .59  compared 
to  the  goal  of  .75,  than  the  specified  level. 

To  compare  strictly  using  the  system  average  is  potentially  misleading.  The 
variation  between  subsystems  is  important  to  look  at.  A  subsystem  average  may 
be  quite  low  and  in  fact  be  a  weak  link,  in  terms  of  quality,  within  the 
system.  The  same  analogy  applies  at  a  module  level.  The  Exception  Report 
provides  the  capability  within  AMT  to  identify  those  modules  which  are 
potential  problem  modules.  The  metric  which  varied  greatest  within  the  system 
was  the  complexity  measure.  This  metric  was  used  to  monitor  the  development 
team  and  help  during  design  and  code  walkthroughs  to  control  the  complexity  of 
the  design.  Because  the  metric  values  were  not  contractual  requirements, 
redesign  and  reimplementation  was  not  done  strictly  to  improve  the  metric 
value  but  only  done  when  the  complexity  was  obviously  too  high  and  would  have 
a  major  impact  on  the  quality  of  the  software. 

At  the  metric  level,  the  AMT  development  team  met  8  of  the  11  (or  73%)  of  the 
specified  goals. 

Another  view  is  illustrated  in  Table  5. 4. 2. 3-2,  where  those  metrics  related  to 
each  quality  factor  and  how  well  the  software  scored  in  terms  of  either  the 
thresholds  established  or  past  experience  is  shown.  In  the  case  of  metrics 
for  which  a  threshold  value  was  not  established,  the  metric  score  was  compared 
with  past  experience.  The  table  identifies  if  the  values  achieved  for  the  AMT 
were  low  (L),  slightly  higher  (M),  or  much  higher  (H)  than  past  experience. 
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REUSEABILITY 
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INTEROPERABILITY 

N  NM  NM 

j 

CORRECTNESS 


Using  this  view,  the  performance  of  the  development  team.  In  terms  of  the 
metrics  relative  to  the  six  quality  factors  Identified  as  Important,  can  be 
assessed.  For  example,  for  the  quality  factor,  portability,  four  metrics 
exceeded  the  specified  threshold  values,  one  metric  for  which  a  threshold  was 
not  specified,  scored  slightly  better  than  past  experience,  and  only  one 
metric  did  not  achieve  the  specified  threshold  value.  Thus  from  a  portability 
viewpoint,  five  out  of  six  (83%)  of  the  metrics  related  to  portability 
exceeded  expectations. 

Using  the  metrics  In  this  way,  the  development  team  could  be  assessed  as 
having  met  83%  (5  of  6)  of  their  goals  related  to  portability,  86%  (6  of  7) 
related  to  flexibility,  86%  (6  of  7)  related  to  reusability,  0%  (0  of  1)  for 
interoperability,  25%  (1  of  4)  for  correctness,  and  60%  (6  of  10)  for 
maintainability. 

The  NM  indicator  In  the  table  Identifies  those  metrics  not  measured.  In  most 
cases,  these  metrics  were  not  measured  because  without  automated  support.  It 
was  not  possible  to  measure  them  against  any  of  the  AMT  source  code. 

In  some  cases,  data  or  material  was  not  available  for  measuring  that 
particular  metric.  An  example  of  this  situation  Is  the  Data  Communication 
(DC.l)  metric.  No  other  tool  was  identified  to  be  Interfaced  with  the  AMT  so 
no  consideration  was  given  to  how  compatible  the  AMT  data  structure  was  with 
any  other  tool. 

Table  5. 4. 2. 3-3  provides  the  results  of  substituting  the  average  metric  scores 
for  the  system  into  the  normalization  functions.  These  normalization 
functions.  Including  the  Individual  metric  functions  as  well  as  the 
multivariate  functions,  are  defined  In  the  Software  Quality  Measurement  Manual 
[MCCJ  79], 

Because  the  Modular  Implementation  Measure  (MO. 2)  was  measured  differently 
than  previous  studies,  a  constant  of  .25  was  added  to  the  MO. 2  metric  value  to 
arrive  at  a  corrected  normalization  function  rating.  This  correction  constant 
was  only  used  for  the  normalization  functions  that  contained  the  MO. 2  metric. 
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In  addition  a  correction  to  the  normalization  function  for  maintainability  was 
made.  This  correction  is  to  account  for  using  a  different  complexity  measure, 
SI. 3.  In  the  Factors  in  Software  Quality  study  [MCCJ77a],  Halstead's  E 
measure  [HALM77]  was  used.  In  the  Metrics  Enhancement  [MCCJ79]  and  the  AMT 
development,  McCabe's  [MCCT76]  metric  was  used.  This  change  was  not  taken 
into  account  in  the  normalization  function  documented  In  the  Software  Quality 
Measurement  Manual.  To  account  for  the  different  measure,  a  factor  of  2.46 
multiplied  by  the  complexity  measure  should  be  substituted  for  SI. 3  in  the 
normalization  function  shown  in  Table  5.4. 2. 3-3  to  arrive  at  the  calculated 
score.  In  the  future,  the  normalization  function  recommended  is: 

-  .2  +  1 . 5M ( S I . 3 )  +  . 14M(M0.2)  +  ,33M(S0.2)  *  rffl 

when  using  McCabe's  metric  as  the  complexity  metric  (SI. 3). 

The  resultant  ratings  are  compared  with  the  established  goals  in 
Table  5.4. 2. 3-3.  The  individual  factors  are  discussed  below. 

i 

PORTABILITY 

The  AMT  software  is  considered  highly  portable.  The  system  software  dependent 
and  machine  dependent  software  have  been  minimized  and  localized.  The  metric 
scores  for  these  measures  and  others  related  to  Portability  are  all  relatively 
high  except  for  the  Modular  Implementation  Measure,  MO. 2. 

The  goal  identified  for  portability  was  .75.  The  Software  Quality  Measurement 
Manual  states  that  this  rating  Is  equivalent  to  1  -  (effort  to  transfer  )/ 
(effort  to  implement).  Thus  a  goal  of  .75  is  the  same  as  saying  the  effort  to 
transport  a  module  of  the  AMT  to  another  system  should  take  25%  or  less  of  the 
effort  to  implement  that  module.  The  score  achieved  by  calculating  the 
normalization  function  Is  equivalent  to  the  rating.  Thus  a  normalization 
function  value  of  .9  equates  to  a  rating  of  .9  and  means  the  software  can  be 
transported  to  another  system  In  10%  or  less  of  the  effort  required  to 
Implement  the  software. 

The  multivariate  function  for  portability  and  the  individual  normalization 
functions  for  all  but  the  SI.l  metric  exceeded  the  specified  goal  of  .75.  The 


- 1 
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portability  of  the  AMT  was  demonstrated  by  the  relative  ease  with  which  the 
initial  prototype  version  of  the  AMT,  developed  on  the  VAX  11/780,  was 
transported  to  the  RAOC  H6180.  The  value  of  1.6  should  be  interpreted  as  1. 

Because  the  previous  effort  [McCJ79]  to  establish  beta  coefficients  was  based  ~fT 
upon  a  limited  sample  of  projects  and  the  dependent  variable  of  portability  is 
somewhat  illusive,  the  regression  equations  need  to  be  adjusted  to  reflect  a 
more  accurate  representation.  This  will  require  additional  portability  data 
from  a  wider  range  of  projects  to  be  analyzed. 

FLEXIBILITY 

The  AMT  software  exhibited  characteristics  that  indicate  it  will  be  highly 
flexible.  The  metrics  related  to  flexibility  were  all  relatively  high  as  -# 

shown  in  Table  5. 4. 2. 3-1.  A  generalized  parser  was  utilized  in  the  Automated 
Measurement  Services  Subsystems  to  facilitate  modifying  the  AMT  to  process 
other  programming  languages  besides  COBOL.  The  grammar  description  of  COBOL 
developed  was  at  a  high  enough  level  of  abstraction  to  handle  a  wide  number  of  IB 
COBOL  grammars  while  still  measuring  the  needed  characteristics  to  calculate 
the  metrics.  The  normalization  function  calculation  resulted  in  a  value  of 
.56.  This  value  relates  to  the  average  amount  of  effort  it  takes  to  make  a 
modification  to  the  software  based  on  a  change  in  requirements.  The  • 

relationship  is  1/.56  =  the  average  person  days  to  make  a  modification,  or 
1.78  person  days.  The  rating  for  flexibility  equals  1  -  ,05x(average  person 
days  to  modify).  The  rating  therefore  is  .91. 

i 

REUSEABILITY 

The  software  exhibited  high  scores  for  those  metrics  related  to  reuseability. 

The  interfaces  and  functional  decomposition  of  the  system  are  well  defined  to 
facilitate  reuse.  As  shown  in  Table  5.4. 2. 3-1  and  Table  5. 4. 2. 3-2,  six  of  the  9 

seven  metrics  related  to  reuseability  exceeded  expectations.  These 
expectations  were  based  on  threshold  values  or  past  experience.  In 
particular,  the  three  metrics  related  to  comments  (0S.1,  SD.2,  SD.3),  the 
machine  Independence  metric  (MI . 1 ) ,  and  the  implementation  generality  metric  i 

(GE.2),  all  exceeded  the  threshold  values  specified.  The  system  software 
independence  metric  (SS.l)  was  higher  than  the  values  experienced  during  the 
Metric  Enhancement  study.  The  Modular  Implementation  metric  (MO. 2)  was  the 
only  reuseabllity-related  metric  that  did  not  meet  or  exceed  the  threshold  • 

value  specified. 
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INTEROPERABILITY 

While  the  metrics  needed  to  assess  this  quality  factor  were  not  measured, 
facilites  to  allow  interfacing  with  the  AMT  were  built  into  the  system.  The 
PUT  and  GET  commands  can  be  utilized  to  interact  with  the  AMT  data  base.  They 
could  be  used  to  extract  pertinent  data  from  output  of  an  existing  software 
tools  and  placed  into  the  AMT  data  base.  The  only  metric  measured  that  was 
related  to  interoperability  was  MO. 2  which  has  been  discussed  already. 

CORRECTNESS 

The  metrics  related  to  this  quality  factor  were  mixed  in  their  performance  at 
best.  Only  one  metric,  CS.2,  had  a  specified  threshold  value  for  the  AMT 
development.  That  metric  exceeded  the  threshold.  Three  other  metrics  related 
to  correctness  did  not  achieve  values  as  high  as  those  of  past  studies. 

The  interpretation  that  can  be  made  is  that  the  previous  studies  involved 
taking  the  measurements  from  existing  operational  systems  which  you  would 
expect  to  be  more  mature,  have  more  complete  documentation,  and  therefore 
achieve  higher  metric  scores  than  the  AMT. 

The  testing  process  used  five  COBOL  programs  provided  from  a  production  system 
at  the  USACSC.  The  Test  Plan  (CORL  A0015)  and  the  Test  Analysis  Report  (CDRL 
A014)  describe  the  test  process.  All  planned  tests  except  one  were 
successfully  accomplished.  One  functional  capability  not  provided  was  the 
alternate  print  capability. 

MAINTAINABILITY 

The  comments,  structure,  implementation  techniques,  and  control  flow 
complexity  were  controlled  during  the  development,  and  these  practices  were 
reflected  in  the  metric  scores.  The  normalization  function  using  the 
multiplication  factor  discussed  previously  for  the  complexity  metric  (SI. 3) 
and  the  addition  factor  for  the  MO. 2  metric  resulted  in  a  value  of  .26.  This 
equates  (1/.26)  to  an  average  of  3.8  person  days  to  fix  an  err^r  in  the 
software.  The  rating  then  is  1  -  .lx  (average  effort  to  fix  an  error)  or 
.62.  This  is  slightly  lower  than  the  .7  rating  or  goal  specified.  The 
complexity  of  the  system  was  slightly  higher  than  desired  and  resulted  in  the 
slightly  lower  rating. 
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RELIABILITY 

Reliability  was  not  a  quality  factor  specified  as  critically  important  to  the 
AMT  because  it  is  basically  a  prototype  system.  However,  for  evaluation 
purposes,  we  monitored  the  performance  of  the  metric  scores  related  to  the 
reliability  normalization  function.  The  calculated  normalization  function 
value  was  .45.  The  rating  is  calculated  by  doubling  this  value  to  .9  and  is 
equated  to  1  -  (number  of  errors) /( 100  lines  of  code).  The  Industry  average 
is  approximately  2  errors  per  100  lines  of  code  or  .98.  The  .9  achieved  by 
the  AMT  development  met  the  goal  specified. 

5.5  COMPARISON  OF  AMT  METRIC  SCORES  WITH  PAST  EXPERIENCES 
Table  5.5-1  provides  a  comparison  of  the  average  metric  scores  for  the  AMT 
with  past  experiences.  These  past  experiences  Include  the  JOVIAL  Command  and 
Control  System  used  during  the  Factors  In  Software  Quality  Contract  (MCCJ77), 
the  Management  Information  System  (MARDIS)  written  in  COBOL  and  The  Software 
Support  System  written  in  FORTRAN  that  were  used  in  the  Metrics  Enhancement 
Contract  (MCCJ79),  a  Data  Base  Management  System  written  in  JOVIAL,  and  a 
Telemetry  Prediction  Simulation  System  written  in  JOVIAL.  The  AMT  was  written 
in  a  structured  FORTRAN.  The  annotation  MNM"  in  the  table  Indicates  a  metric 
that  was  not  measured. 

The  following  metrics  had  scores  higher  for  the  AMT  than  past  experiences: 

CO. 1  Conciseness 

ET. 1  Error  Tolerance 

GE.2  Generality 

MI. 1  Machine  Independence 

SD.3  Descriptiveness  of  Language 

SS.l  System  Software  Independence 

These  metrics  Indicate  the  concern  primarily  for  portability  and  flexibility 
during  the  AMT  development. 
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Table  5.5-1 

Implementation  Metric  Score  Comparisons 


METRIC 

AMT 

COBOL 

AVERAGE 

JOVIAL 

MIS 

JOVIAL 

JJVIAL 

SCORE 

C2 _ 

FORTRAN 

DBMS  EXS 

ACCURACY 

AY.l 

.29 

NM 

NM 

NM 

NM 

CONCISENESS 

CO.l 

.996 

.78 

.12 

.75 

.60 

COMPLETENESS 

CP.l 

.25 

.92 

NM 

NM 

NM 

PROCEDURE  CONSISTENCY 

CS.l 

.76 

.99 

NM 

NM 

NM 

DATA  CONSISTENCY 

CS.2 

.39 

.8 

.68 

NM 

NM 

ITERATIVE  PROCESSING  EFFICIENCY 

EE. 2 

.51 

.67 

.50 

NM 

NM 

DATA  USAGE  EFFICIENCY 

EE. 3 

.59 

.96 

.85 

NM 

NM 

ERROR  TOLERANCE  CONTROL 

ET.l 

.77 

.77 

.NM 

NM 

NM 

ERROR  TOLERANCE  INPUT  DATA 

ET.2 

.25 

.84 

.02 

NM 

NM 

ERROR  TOLERANCE  COMPUTATION 

ET.3 

.07 

.51 

.07 

NM 

NM 

DATA  STORAGE  EXPANDABILITY 

EX.l 

.12 

NM 

NM 

NM 

NM 

COMPUTATIONAL  EXTENSIBILITY 

EX. 2 

.05 

NM 

.07 

NM 

NM 

IMPLEMENATION  GENERALITY 

GE.2 

.74 

.48 

.35 

.12 

.12 

MACHINE  INDEPENDENCE 

MI.l 

.84 

.13 

.21 

NM 

NM 

MODULAR  IMPLEMENTATION 

MO.  2 

.37 

.68 

.71 

NM 

NM 

QUANTITY  OF  COMMENTS 

SD.l 

.46 

.69 

.35 

.38 

.35 

EFFECTIVE  OF  COMMENTS 

SD.2 

.58 

.74 

.40 

NM 

NM 

DESCRIPTIVENESS  OF  LANGUAGE 

SD.3 

.90 

.82 

.57 

NM 

NM 

DESIGN  STRUCTURE 

SI.l 

.59 

.64 

.87 

NM 

NM 

COMPLEXITY 

SI. 3 

.12 

.66 

.23 

.10 

.08 

CODE  SIMPLICITY 

SI. 4 

.60 

.76 

.57 

.66 

.73 

SYSTEM  SOFTWARE  INDEPENDENCE 

SS.l 

.38 

.18 

.01 

.03 

.12 

TRACEABILITY 

TR.l 

.07 

1 

NM 

NM 

NM 

NORMALIZATION  FUNCTION  (ratlnqs) 

PORTABILITY 

1 

NM 

.23 

NM 

m 

FLEXIBILITY 

.91 

.88 

.86 

NM 

NM 

MAINTAINABILITY 

.62 

.68 

.9 

NM 

NM 

RELIABILITY 

.9 

.98 

.96 

NM 

NM 

NM  •  NOT  MEASURED 
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The  following  metrics  had  scores  lower  for  the  AMT  than  past  experiences: 


CP.l  Completeness 

CS.l  Procedure  Consistency 

CS.2  Data  Consistency 

EE. 3  Data  Usage  Efficiency 

MO. 2  Modular  Implementation 
SI.l  Design  Structure 

TR.l  Traceability 


These  metrics  indicate  a  lesser  attention  provided  to  characteristics  related 
to  correctness  and  reliability.  The  scores  of  the  AMT  metrics  were  not  low  in 
the  absolute  sense  but  were  lower  than  those  achieved  in  the  command  and 
control  software  and  other  contract  deliverable  software.  This  is 
understandable  considering  the  AMT  is  a  prototype  research  tool.  Also  shown 
in  the  table  are  the  ratings  achieved  for  the  four  factors  that  have 
established  normalization  functions.  The  relative  ratings  for  the  Factors  in 
Software  Quality  (FSQ)  study,  the  Metrics  Enhancement  (ME)  study,  and  the 
Automated  Measurement  Tool  (AMT)  development  for  these  factors  were  (from  high 
to  low): 


PORTABILITY 

AMT 

FSQ 

ME 


MAINTAINABILITY 

ME 

FSQ 

AMT 


FLEXIBILITY 
AMT 
FSQ 
ME 

5.6  EXPERIMENT  CONCLUSIONS 
As  a  result  of  applying  the  metrics  during  the  development  of  the  AMT  several 
general  observations  can  be  made.  First  the  use  of  the  quality  factors  to 
identify  what  qualities  were  desired  provided  an  excellent  technique  for 
focusing  standards  and  conventions  and  the  goals  of  the  development  team  to 


RELIABILITY 

FSQ 

ME 

AMT 


meet  the  customers  requirements.  Second,  the  use  of  the  metrics  during  the 
development  as  a  development  team  tool  as  well  as  a  mechanism  for  reviewing 
requirements  with  the  customer  proved  effective.  Third,  based  on  the  metrics, 
the  AMT  development  was  reasonably  successful  at  achieving  the  quality  goals 
set  at  the  beginning  of  the  project.  At  the  metric  level,  8  of  11  (83%) 
specified  goals  were  met.  Of  the  three  metric  thresholds  not  achieved,  the 
scores  realized  were  89%,  79%  and  52%  of  the  values  desired.  At  the 
normalization  functon  level,  3  of  4  specified  goals  were  met.  The  one  not  met 
(maintainability)  was  89%  of  the  desired  value. 

There  were  some  negative  aspects  identified.  First,  the  setting  of  the 
specific  quality  goals  was  done  with  relatively  little  experience  data.  In 
some  cases,  such  as  the  modular  implementation  (MO. 2)  where  the  metric 
algorithm  changed  and  the  goal  had  been  set  too  high,  the  goals  established 
were  not  reasonable.  The  setting  of  goals  should  be  carefully  considered  and 
reviewed  between  the  customer  and  development  team.  Second,  continued 
validation  of  the  normalization  functions  is  required.  A  complete  validation, 
i.e.,  statistical  analysis,  of  the  data  should  be  performed  on  new  sets  of 
data  to  gain  more  confidence  in  the  normalization  functions  accuracy.  Third, 
considerable  interaction  between  the  customer  and  the  development  team  is 
needed  to  ensure  effective  use  of  the  quality  feedback  provided  by  the 
metrics.  Tradeoff  analyses  are  necessary  to  ensure  wasted  effort  is  not  spent 
correcting  deficiencies  which  are  not  important  or  measuring  metrics  which  are 
not  critical.  Fourth,  automated  support  was  not  available  and  hindered  the 
effective  daily  use  of  the  metrics  by  the  development  team.  In  general  the 
following  conclusions  can  be  drawn: 

(1)  The  metrics  proved  to  be  an  effective  tool  for  setting  quality  goals, 
identifying  standards  and  conventions  to  guide  the  development,  and 
monitoring  the  progress  toward  these  goals  in-line  with  the 
development. 

(2)  Automated  tools  are  necessary  to  provided  reliable,  timely  metric 
Information. 
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(2)  An  interactive  customer  is  necessary.  More  quantitative  Information 
about  the  software  product  is  available  and  should  be  used. 

(4)  The  normalization  functions  need  continued  validation  before  they  can 
be  generally  used.  They  should  be  validated  and  tailored  to  specific 
applications  and  development  environments. 

(5)  The  metrics  could  be  utilized  as  a  contractual  instrument.  The 
recommendation  is  to  use  them  for  determining  incentive  or  award 
fees.  Their  use  as  an  absolute  acceptance  criteria  is  possible  but 
the  specific  metrics  and  threshold  values  would  have  to  be  negotiated 
prior  to  contract  start. 


SECTION  6 

FUTURE  DEVELOPMENT 


The  AMT  was  developed  to  demonstrate  the  concept  of  automated  collection  and 
reporting  of  software  metrics.  A  minimum  set  of  metrics  are  automatically 
collected  from  COBOL  source  code.  A  fairly  extensive  set  of  reports  are 
generated  to  fulfill  the  requirements  of  a  number  of  personnel  who  might  use 
the  AMT. 

Several  areas  of  the  AMT  could  be  enhanced  for  use  on  an  actual  large  scale 
software  development.  Under  the  category  of  enhancements,  the  following 
aspects  of  the  AMT  could  be  modified  or  added: 

(1)  Add  a  form  entry  system  for  easier  manual  input  of  worksheet  data. 

(2)  Modify  the  Report  Generation  Services  Subsystem  to  be  more  flexible 
in  providing  user  defined  reports. 

(3)  Provide  an  interface  to  a  statistical  package. 

(4)  Interface  AMT  with  other  tools,  especially  tools  that  would  support 
automated  measurement  during  requirements  definition  and  design 
phases. 

(5)  Expand  COBOL  grammar  description  and  Automated  Measurement  Services 
Subsystem  to  support  additional  metrics  automation. 

(6)  Define  another  language  grammar  (eg.  FORTRAN)  to  parser,  develop 
scanner  and  incorporate  processing  capability  for  another  language. 

(7)  Tie  AMT  into  Configuration  Control  and  Error  Reporting  Systems  or 
Program  Support  Libraries. 

(8)  Transport  AMT  to  other  computing  environments. 

(9)  Expand  data  base  caj.  .  . i ties  beyond  50  modules. 
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APPENDIX  A 
SAMPLE  REPORTS 


A-l 


METRIC  WORKSHEET  1 

SYSTEM 

DATE  /JL  -  77 

REQUIREMENTS  ANALYSIS/SYSTEM  LEYEL 

NAME:  Amt 

INSPECTOR:  EAA-TSoMOCQ _ 

I.  COMPLETENESS  (CORRECTNESS,  RELIABILITY) 


1.  Number  of  major  functions  Identified  (equivalent  to  CPCI).  CP.l 

2.  Are  requirements  1 tami zed  so  tnat  the  various  functions  to  be  performed,  their 

ss 

Inputs  and  outputs,  are  clearly  delineated?  CP.l(l) 

N 

3.  Number  of  major  data  references.  CP. 1(2) 

33 

4.  How  many  of  tftese  data  references  are  not  defined?  CP. 1(2) 

O 

5.  How  many  defined  functions  are  not  used?  CP.1(3) 

Cj 

5.  How  many  referenced  functions  are  not  defined?  CP.l (4) 

7.  How  many  data  references  are  not  used?  CP. 1(2) 

O 

3.  How  many  referenced  data  references  are  not  defined?  CP.l (S) 

_ Q _ 

9.  Is  the  flow  of  processing  and  all  decision  points  in  that  flow  described?  CP. 1(5’ 

Y 

H 

10.  How  many  proolem  reports  related  to  the  requirements  have  been  recorded?  CP. 1(7) 

ill.  How  many  of  those  problem  reports  have  been  closed  (resolved)?  CP. 1(7) 

! 

|  II.  PRECISION  (RELIABILITY) 

j  1.  Has  an  error  analysis  been  performed  and  budgeted  to  functions?  AY.1(1) 

Y 

Fol 

2.  Are  there  definitive  statements  of  the  accuracy  requirements  for  inputs. 

i  outputs,  processing,  and  constants?  AY. 1(2) 

V 

gal 

J 

;  3.  Are  there  definitive  statements  of  the  error  tolerance  of  inout  data?  ET.2(1) 

Y 

PHI 

j 

1  4.  Are  there  definitive  statements  o'  the  requirements  for  recovery  from 

■ 

computational  failures?  ET.3(1) 

H 

Wi 

5.  Is  there  a  definitive  statement  of  the  requirement  for  recovery  from  hardware 

faults?  ET.4(!) 

7 

5.  Is  there  a  definitive  statement  of  the  requirements  for  recovery  from  device 

errors?  ET.S(l) 

Y 

III.  SECURITY  (INTEGRITY) 

1.  Is  there  a  definitive  statement  of  th«  requirements  for  user  Input/outout 
access  controls?  AC . 1 ( 1 ) 

2.  Is  there  a  daflni tlve  statement  of  the  requirements  for  data  Sase  accsss 
controls?  AC. 1(2) 

3.  Is  there  a  definitive  statement  of  the  requirements  'or  memory  orotactlcn 
across  tasks’  AC.  1(3) 

<i.  Is  there  a  definitive  statement  of  the  reoulrements  'or  recordlno  and 
reoortlng  accsss  to  system?  AA.l(l) 

S.  Is  there  a  dsflnltlvs  statement  of  trts  rsouirsmsnts  for  immediate 
indication  of  accsss  violation?  AA.1(2) 


\0 


i  >  Yl 


& 


METRIC  WORKSHEET  1 
REQUIREMENTS  ANALYSIS/SYSTEM  LEY EL 


SYSTEM 

NAME: 


OATE _ 

INSPECTOR: 


IV.  HUMAN  INTERFACE  (USABILITY  } 


1.  Are  all  steps  In  the  operation  described  (operations  concept)?  QP.l(l) 

2.  Are  all  error  conditions  to  be  reported  to  operator/user  Identified  and 
the  responses  described?  OP. 1(2) 

3.  Is  there  a  statement  of  the  requirement  for  the  capability  to  Interrupt 
operation,  obtain  status,  modify,  and  continue  processing?  OP.  1(3) 

<l.  Is  there  a  definitive  statement  of  requirements  for  optional  Input  media? 

CM* 1(6) 

5.  Is  there  a  definitive  statement  of  requirements  for  optional  output  media? 

CM. 2(7) 

6.  Is  there  a  definitive  statement  of  requirements  for  selective  output 
control?  CM. 2(1 ) 


V.  PERFORMANCE  (EFFICIENCY) 


1.  Have  performance  requirements  (storage  and  run  time)  been  identified  for 
the  functions  to  be  performed?  EE.l 


VI.  SYSTEM  INTERFACES  ( INTEROPERAB ILITY ) 


1.  Is  there  a  definitive  statement  of  the  requirements  for  coneunl cation  with 
other  systems?  CC.1(1) 

2.  Is  there  a  definitive  statement  of  the  requirements  for  standard  data 
representations  for  comnunl  cation  with  other  systems?  DC.  1(1) 


IB 


VII.  INSPECTOR'S  COW  ENTS 


Make  any  general  or  specific  comments  that  relate  to  the  quality  observed  while 
applying  this  checklist. 


METRIC  WORKSHEET  2A 
OESIGM/SYSTEM  LEVEL 


DATE;  _ 

INSPECTOR: 


I.  COMPLETENESS  (CORRECTNESS.  RELIABILITY) 


Is  there  a  matrix  relating  Itemized  requirements  to  modules  which  implement 
those  requirements?  TR.l 

How  many  major  functions  (CPCIS)  are  identified?  CP.  1 
How  many  functions  Identified  are  not  defined?  CP. 1(2) 

How  many  defined  functions  are  not  used?  CP. 1(3) 

How  many  Interfaces  between  functions  are  not  defined?  CP. 1(6) 

Number  of  total  problem  reports  recorded?  CP. 1(7) 

Number  of  those  reports  that  have  not  been  closed  (resolved?)  CP. 1(7) 
Profile  of  problem  reports:  (number  of  following  types)  8.  Computational 


Logic 

Input/output 
Data  handling 
OS/ System  Suoport 
Configuration 
Routine/  Routine 
interface 


Profile  of  problem  reports:  (number  of  following  types)  8 
II.  PRECISION  (RELIABILITY)  !? 


Have  math  library  routines  to  be  used  been 
checked  for  sufficiency  with  regards  to 
accuracy  requirements?  AY. I (3) 

Is  concurrent  processing  centrally 
controlled?  ET.  1(1) 

How  many  error  conditions  are  reported 
by  the  system?  ET.1(2) 

How  many  of  those  errors  are  automatically 
fixed  or  bypassed  and  processing  continues? 
How  many,  require  operator  intervention?^) 
Are  provisions  for  recovery  from  hardware' 
faults  provided?  ET.4(2) 

Are  provisions  for  recovery  from  device 
errors  provided?  ET.5(2) 


PORTABILITY.  REUSABILITY.  INTEROPERABILITY) 


Is  a  hierarchy  of  system,  identifying  all  modules  in  the  system  provided? 
Number  of  Modules  SI. 1(2)  MO. 2(1)  SI. 1(1) 

Are  there  any  duplicate  functions?  SI. 1(2) 

3ased  on  hierarchy  or  a  call/cailed  matrix,  now  many  modules  are  called  by 
■more  than  one  other  module?  £.1  MO. 2(1) 

Are  the  constants  used  in  the  system  defined  once?  GE.2(5) 


15. 

Routine/System 

Interface 

16. 

Tape  Processing 

17. 

User  interface 

18. 

data  base  Interface 

19. 

user  requested 
changes 

20. 

Preset  data 

21. 

* 

Global  variable 
definition 

22. 

Recurrent  errors 

23. 

Documentation 

24. 

Requi rement 
compl i ance 

25. 

Operator 

26. 

Quest! ons 

Hardware 

METRIC  rfGRiCjHEn  2A 

SYSTEM 

OATE; 

OESIGN/SYSTEM  LEVEL 

NAME 

INSPECTOR: 

|  IV.  OPTIMIZATION  (EFFICIENCY) 

1.  Are  storage  regul rements  allocated  to  design?  SE.l(l) 

V5£l 

N 

2.  Are  virtual  storage  facilities  used?  SE.1{2) 

N 

3.  Is  dynamic  memory  management  used?  SE . 1 ( 5 ) 

M 

N 

4.  Is  a  performance  optimizing  compiler  used?  EE. 2(2) 

Y 

wm 

S.  Is  global  data  defined  once?  CS.2(3) 

N 

3.  Have  Oata  Sase  or  files  been  organized  for  efficient  processing?  EE. 3(5) 

f&i 

N 

7.  Is  data  packing  used?  EE. 2(5) 

Y 

8.  Number  of  overlays  EE. 2(4) 

9.  Overlay  efficiency  -  memory  allocation  EE. 2 (8) 

10.  max  overlay  size 

11.  min  overlay  size 

V.  SECURITY  (INTEGRITY) 


1.  Are  user  Input/Output  access  controls  provided?  AC.l(l) 

2.  Are  Oata  8ase  access  controls  provided?  AC.1(2) 

3.  Is  memory  protection  across  tasks  provided?  AC. 1(3) 

4.  Are  there  provisions  for  recording  and  reporting  errors?  AC. 2(1 ,2) 

m i 

N 

tt 

hl 

N' 

KM 

N 

! 

VI.  SYSTEM  INTERFACES  (INTEROPERABILITY) 

1.  How  many  other  systems  will  this  system  interface  with?  CC.1(1) 

2.  Have  protoc  1  standards  been  established?  CC.1(2) 

3.  Are  they  being  complied  with?  CC.1(2) 

4.  Number  of  modules  used  for  Input  and  output  to  other  systems?  CC. 1(3,4) 

5.  Has  a  standard  data  representation  been  established  or  translation 
standards  between  representations  been  established?  OC.l(l) 

6.  Are  they  being  compiled  with?  0C.1(2) 

7.  Number  of  modules  used  to  perform  translations?  0C.1(3) 

nSM 

N 

tesi 

N 

\ 

N 

rai 

mm 

i 

VII.  HUMAN  INTERFACE  (USABILITY) 

1.  Are  all  steps  in  operation  described  Including  alternative  flows?  0P.1(1) 

2.  Number  of  operator  actions?  OP.  1(4) 

Y  1  N 

-Mr-., 

1.  Art  all  steps  In  operation  described  Including  alternative  flows?  OP .  1  ( 1 ) 

2.  Number  of  operator  actions?  OP.  1(4) 


METRIC  WORKSHEET  2A 
OESIGN/ SYSTEM  LEVEL 


SYSTEM 

NAME: 


DATE: 

INSPECTOR 


VII.  HUMAN  INTERFACE  (USABILITY)  (Continued) 


3.  Estimated  or  Actual  time  to  perform?  OP. 1(A) 

4.  Budgeted  time  for  complete  job?  OP .1(4) 

5.  Are  job  set  up  and  tear  down  procedures  described?  OP. 1(5) 

Y 

— 

N 

6.  Is  a  hard  copy  of  operator  Interactions  to  be  maintained?  OP. 1(6) 

— Y _ 

N 

7.  Number  of  operator  messages  and  responses?  0P.1?2) 

8.  Number  of  different  formats?  OP.  1(2) 

9.  Are  all  error  conditions  and  responses  appropriately  described?  OP. 1(2) 

Y 

N 

10.  Ooes  the  capability  exist  for  the  operato*  to  Interrupt,  obtain  status. 

save,  modify,  and  continue  processing?  OP. 1(3) 

Y 

N 

11.  Are  lesson  plans/training  materials  for  operators,  end  users,  and 

■  ■ 

malntainers  provided?  TN.1(1) 

Y 

H 

12.  Are  realistic,  simulated  exercises  provided?  TN.1(2) 

Y 

N 

13.  Are  help  and  diagnostic  information  available?  TN . 1 ( 3) 

Y 

N 

14.  Number  of  input  formats  CM. 1(2) 

1 

15.  Number  of  input  values  CM. 1(1) 

16.  Number  of  default  values  CM.  1(1) 

1 

17.  Number  of  self-identifying  Input  values  CM.1(3) 

1 

18.  Can  input  be  verified  by  user  prior  to  execution?  CM.  1(4) 

Y 

N 

1 

19.  Is  input  terminated  by  explicitly  defined  by  logical  and  of  input?  CM.1(5 

Y 

_ 1 _ 

1 

20.  Can  input  be  specified  from  different  media?  CM.  1(6) 

Y 

m 

21.  Are  there  selective  output  controls?  CM. 2(1) 

Y 

N 

1 

22.  Oo  outputs  have  unique  descriptive  user  oriented  labels?  CM. 2(5) 

Y 

n 

1 

23.  Oo  outputs  have  user  oriented  units?  CM. 2(3) 

Y 

N 

1 

24.  Number  of  output  formats?  CM. 2(4 ) 

1 

25.  Are  logical  groups  of  output  separated  for  user  emaml natl on ?  CM. 2 ( S ) 

Y 

N 

I 

26.  Are  relationships  between  error  messages  and  outputs  unambiguous?  CM. 2(6) 

Y 

N 

1 

27.  Are  there  provisions  for  directing  output  to  different  media?  CM.2(7) 

i 

N 

1 

VII!.  TESTING  (TESTABILITY)  APPLY 

TO  TEST  PUN,  PROCEDURES,  RESULTS 

1.  Number  of  paths?  IN.l(l) 

4.  Number  of  input  parameters  to 

2.  Number  of  paths  to  be  tested?,,  . , 

be  tested?  IN. 1(2) 

IN .  1  ( 1 . 

3.  Number  of  input  parameters? 

IN. 1(1 i 

5.  <umber  of  Interfaces?  IN. 2(1) 

•  ' 
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UNCLASSIFIED 


F/G  9/2 


NL 


METRIC  WORKSHEET  2A 
0ESI5N/SYSTSM  LEVEL 


SYSTEM 

NAME; 


DATE: 

INSPECTOR;. 


VIII.  TESTINS  (TESTABILITY)  -  APPLY  TO  TEST  PLAN.  PROCEDURES,  RESULTS  (CONTINUED)  J 


Make  any  general  or  specific  corments  about  the  quality  observed  while  applying  this 
checklist. 


i;  Can  you  clearly  distinguish  inputs,  outputs,  and  the  function  being  performed?  CP.  1(1) 

Z.  How  many  data  references  are  not  defined,  computed,  or  obtained  from  an  external 
source?  CP. 1(2) 


3.  Are  all  conditions  and  processing  defined  for  each  decision  point?  CP. 1(5) 

4.  How  many  problem  reports  have  been  recorded  for  this  module?  CP. 1(7) 


Profile  of  Problem  Reports: 

^  Number  of  problem  reports  still  outstanding  CP.1(7^ 

II.  PRECISION  (RELIABILITY) 

1.  When  an  error  condition  is  detected,  is  it 
passed  to  calling  module?  ET.1(3) 

2.  Have  numerical  techniques  being  used  In  algori¬ 
thm  been  analyzed  with  regards  to  accuracy 
requirements?  AY. 1(a) 

3.  Are  values  of  Inputs  range  tested?  £T.2(2) 

-  Are  conflicting  requests  and  Illegal  combina¬ 
tions  identified  and  checked?  ET.2(3) 


Is  there  a  check  to  see  if  all  necessary  data 
is  available  before  processing  begins?  ET .2(5) 


Is  all  input  checked,  reporting  all  errors, 
before  processing  begins?  ET.2(4) 


7.  Are  loop  and  multiple  transfer  Index  parameters 
range  tested  before  use?  ET.3(2) 


3.  Are  subscripts  range  tested  before  use?  ET.3(3) 

9.  Are  outputs  checked  for  reasonableness  before 
processing  continues?  ET.3(4) 


HI.  STRUCTURE  (RELIABILITY,  MAINTAINABILITY,  TESTABILITY) 


Si  Computational 

6.  Logic 

y  Input/Output 

System/OS  Support 

.  Configuration 

|0  Routine/Routine  Inter- 
•  face 

II.  Routine/System  Inter¬ 
face 

l3L.Tap*  Processing 
|3. User  Interface 
H.  Data  Base  Interface 
IS. User  Requested  Changes 
Preset  Data 

| ri  Global  Variable  Defi- 
'*  nition 

|g.  Recurrent  Errors 
Documentation 

20.  Requ i rement  Compliance 

21.  Operator 
21.  Questions 
23  . Hardware 


1.  How  many  Oecislon  Points  are  there? 

SI. 3 

2.  How  many  subdecision  Points  are 
there?  SI .  3 


3.  Hew  many  conditional  branches  are 
there?  SI. 3 

4.  How  many  unconditional  branches 
are  there?  SI. 3 


metric  worksheet  23 

QESIGN/MODULE  LEVEL 


SYSTEM  NAME: 
MOOULE  NAME: 


Pg.  2 


III. 

5. 


6. 


IV. 


1. 


*  2. 


3. 


A. 


5. 


6. 


7. 


V. 


1. 


2. 


3. 


VI. 

77" 


STRUCTURE  (RELIABILITY,  MAINTAINABILITY,  TESTABILITY)  (CONTINUED) 


Is  the  module  dependent  on  the  j 

source  of  the  input  or  the  ^  !  y 

destination  of  the  output?  SI. IGA  1  Y  1 

n 

7.  Are  any  limitations  of  the  proces¬ 
sing  performed  By  the  module 
identified?  EX-20) 

Q 

Is  the  module  dependent  on  know-  S*\ 

ledge  of  prior  processing?  SI. 1(3) 

i 

m 

■ 

8.  Number  of  entrances  into 

9.  Number  of  exits  from  rnodulg^ 

i 

T~ 

REFERENCES  (MAINTAINABILITY ,  FLEXIBILITY,  TESTABILITY,  PORTABILITY.  REUSABILITY. 
INTEROPERABILITY) 


Number  of  references  to  system 
library  routines,  utilities  or  other 
system  provided  facilities  SS.l(l) 

Number  of  Input/output  actions 

MI. 1(2) 

Number  of  calling  sequence  parameters 

MO. 2(3) 

How  many  calling  sequence  parameters 
are  control  variables  MO. 2(3) 

Is  input  passed  as  calling  sequence 
parameters  MO. 2(4) 

m 

3 

WM 

H 

15 

Is  output  passed  back  to  calling 
module?  MO. 2(5) 

Is  control  returned  to  calling 
module  MO. 2(6) 

H 

N 

HI 

8. 


9. 


10. 


11. 


12. 


13. 


14. 


Is  temporary  storage  shared  with 
other  modules?  MO. 2(7) 


Ooes  the  module  mix  input,  out- 
put  and  processing  functions  in 
same  module?  Gc.2(l) 


Number  of  machine  dependent 
functions  performed?  GE.2(2) 

Is  processing  data  volume  limited 

GE.2(3) 

Is  processing  data  value  limited? 

GE.2(4) 


Is  a  coinnon ,  standard  subset  of  ; 
programing  language  to  bemused?  jjj  j 

Is  the  programing  language 
available  in  other  machines? 
_  HI.  1(1) 


EXPANDABILITY  (FLEXIBILITY) 


Is  logical  processing  independent  of  storage  specification?  EX. 1C) 
Are  accuracy,  convergence,  or  timing  attributes  parametric? 

Is  module  table  driven?  EX.2(2) 


OPTIMIZATION  (EFFICIENCY) 


Are  specific  performance  requirements 
module?  EE. 1 


(storage  and  rvnlime}  allocated  to  this 


’4 


1ETRIC  WORKSHEET  2B 
:esig»/mooule  level 


OPTIMIZATION  EFFICIENCY  (CONTINUED 


l.  Which  category  does  processing  fan  in:  EE. 2 


C\*CL*  on£ 


Real-time 

On-line 

Time-constrained 
Non -time  critical 


3.  Are  non-loop  dependent  functions  kept  out  of  loops?  EE. 2(1) 
Is  bit/byte  packing/unpacking  performed  in  loops?  EE. 2(5) 

5.  Is  data  indexed  or  reference  efficiently?  EE. 3(5) 


/II.  functional  categorization 


Categorize  function  performed  by  this  module  according  to  following:  Cxrcie  omc  telew 
|  ^  |  CONTROL  -  an  executive  module  whose  prime  function  is  to  Invoke  other  modules. 


£  INPUT/OUTPUT  -  a  module  whose  prime  function  is  to  conmunlcate  data  between- 
1 - 1  the  computer  and  the  user. 

3  PRE/POSTPROCESSOR  -  a  module  whose  prime  function  is  to  prepare  data  for  or 
*  •  after  the  invocation  of  a  computation  or  data  management 

module. 

|  4  1  ALGORITHM  -  a  module  whose  prime  function  is  computation. 

I  OATA  MANAGEMENT  -  a  module  whose  prime  function  is  to  control  the  flow  of 
/  -I  data  within  the  computer. 

L  SYSTEM  •  a  module  whose  function  is  the  scheduling  of  system  resources  for 
‘ — —  — *  other  modules. 


VIII.  CONSISTENCY 


Does  the  design  representation  comply  with  established  standards  CS . 1 ( 1 ) 
Qo  Input/output  references  comply  with  established  standards  CS.1(3) 

Oo  calling  sequences  comply  with  established  standards  CS.1(2) 

Is  error  handling  done  according  to  established  standards  CS.1(4) 
ire  variable  named  accordino  to  established  standards  CS . 2 ( 2 ) 


IB 

I 


«TRIC  WORKSHEET  3 
SOURCE  COOE/MOOULE  LEVEL 


SYSTEM  NAME: 

MODULE  NAME: 


STRUCTURE  (RELIABILITY,  MAINTAINABILITY,  TESTABILITY) 
Number  of  lines  of  code  MO. 2(2)  lad  I  11.  Nu 


Number  of  lines  excluding  comments 

SI. 4(2) 

Number  of  machine  level  language 
statements  SO. 3(1) 

Number  of  declarative  statements 

SI. 4 

Number  of  data  manipulation  state¬ 
ments  SI.  4 

Number  of  statement  labels  ST.4(6) 
(Do  not  count  format  statements;' 
Number  of  entrances  into  module- 

SI. 1(5) 

Number  of  exits  fronr  module 

SI. 1(5) 

Maximum  nesting  level  SI,4(7) 

Number  of  decision  ooints 

(IF,  WHILE,  REPEAT,  00,  CASE)  SI-3 


11.  Number  of  sub-decision  points 

SI. 3 

12.  Number  of  conditional  branches 
(computed  go  to)  Sl.4(3) 

13.  Number  of  unconditional  branches 
(GOTO,  ESCAPE)  SI. 4(9) 

14.  Number  of  loops  (WHILE,  00) 

SI. 4(3) 

15.  Number  of  loops  with  jumps  out  of 
loop  SI. 4(3) 

16.  Number  of  loop  indlcies  that  are 
modified  SI. 4(4) 

17.  Number  of  constructs-  that  perform 
module  modification  (SWITCH, 

ALTER)  SI. 4(5) 

IB.  Number  of  negative  or  complicated 
compound  boolean  expressions 

19.  Is  a  structured  language  useci^  / 

SI. 2  I 

20.  Ts  flow  top  to  bottom  (are  there  i 
any  backward  branching  GOTOs )si .4( i*" 


CONCISENESS  (MAINTAINABILITY)  -  SEE  SUPPLEMENT 


- r- 

Number  of  operators-  C0.1  j 

an 

3.  Number  of  Operands  CO.T 

2.  Number  of  unique  operators  CO.T 

a 

i 

4.  Number  of  unique  operands  C0.1 

III.  SELF-OESCRIPTIVENESS  (MAINTAINABILITY,  FLEXIBILITY,  TESTABILITY,  PORTABILITY,  REUSABILIT 


1.  Number  of  lines  of  comments  S0.1 


Number  of  non-blank  lines  of  conments 

S0.1 

Are  there  prologue  conments  provided 
containing  information  about  the 
function,  author,  version  number, 
date,  inputs,  outputs,  assumptions 
and  limitations? 

Is  there  a  comnent  which  indicates 
what  itemized  requirement  is 
satisfied  by  this  module?  SftfftT) 

TA.| 

How  many  decision  points  and  trans¬ 
fers  of  control  are  not  conmented? 

SO. 2(3) 

Is  all  machine  language  code  com¬ 
mented?  SO. 2(4) 


7.  Are  non-standard  HOL  statements 
commented?  SO. 2(5) 


8.  How  many  declared  variables  are 
not  described  by  conments? 

SO. 2(6) 

9.  Are  variable  names  (mnemonics) 
descriptive  of  the  physical  or 
functional  property  they 
represent?  SO. 3(2) 

10.  Oo  the  conments  do  more  than 
repeat  the  operation?  SO. 2(7) 


Is  the  code  logically  blocked  and 
indented?  SO. 3(3) 


12.  Number  of  lines  with  more  than 
1  statenw-t.  SO. 3(4) 


!| 

SI 


HETRIC  WORKSHEET  p 

■EBMfc— ■ ■ 

-SOURCE  COOE/MOOULE  LEVEL 

IV.  INPUT/OUTPUT  (RELIABILITY,  FLEXIBILITY,  PORTABILITY) 


r.  Number  of  input  statements  MI.1(2) 

I.  Number  of  output  statements  MI. 1(2) 

3.  Is  amount  of  input  that  can  be 
handled  parametric?  G£.2(3) 

rm 

4.  Are  inputs  range-tested  (for 
inputs  via  calling  sequences, 
global  data,  and  input 
statements)  ET.2(2) 

5.  Are  possible  conflicts  or  Illegal 
combinations  in  inputs  checked? 

ET.2(3) 

3 

3 

D 

m 

m 

6.  Is  there  a  check  to  determine  if 
all  data  is  available  prior  to 
processing?  ET.2(5) 

S 

./.  REFERENCES  (RELIABILITY,  MAINTAINABILITY.  TESTABILITY.  FLEXIBILITY.  PORTABILITY,  REUSABILI' 


T. 

? 


3. 


A. 


3. 


Number  of  calls  to  other  modules 

MO. 2(1) 

Number  of  references  to  system 
library  routines,  utilities,  or 
other  system  provided  functions 

SS.l(l) 

Number  of  calling  sequence  parameters 

MO. 2(3) 

How  many  elements  In  calling 
sequences  are  not  parameters? 

MO. 2(3) 

How  many  of  the  calling  parameters 
(input)  are  control  variables? 

MO. 2(3) 


6. 


7. 


8. 


9. 


How  many  parameters  passed  to  or 
from  other  modules  are  not  defined 
in  this  module?  MO. 2(3) 


Is  input  data  passed  as  parameter? 

MO. 2(4) 


Is  output  data  passed  back  to 
calling  module?  MO. 2(5) 


Is  control  returned  to  calling 
module?  MO. 2(6) 


VI.  OATA  (CORRECTNESS,  RELIABILITY,  MAINTAINABILITY,  TESTABILITY) 


1.  Number  of  local  variables  SI. 4(10) 

2.  Number  of  global  variables  SI. 4(10) 

3.  Number  of  global  variables  renamed 

C&&)  SE.1(3) 


I  o 


4.  How  many  global  variables  are  not 
used  consistently  with  respect  to 
units  or  type?  CS.2(4) 

5.  How  many  variables  are  used  for 
more  than  one  purpose?  CS.2(3) 


VII.  ERROR  HANDLING  -  (RELIABILITY) 


VIII.  (EFFICIENCY) 


Number  of  mix  mode  expressions? 

EE. 3(3)  S&- 

How  many  variables  are  initialized 
when  declared?  EE. 3(2) 

How  many  loops  have  non -loop 
dependent  statements  in  them? EE. 2(1 
How  many  loops  have  b it/by te 
packing/unpacking?  EE. 2(5) 

SE.l (6) 

How  many  compound  expressions 
defined  more  than  once?  EE. 2(3) 


1.  How  many  loop  and  multiple  transfer 
index  parameters  are  not  range 
tested  before  use?  ET.3(2) 

2.  Are  subscript  values  range  tested 
before  use?  ET.3(3) 

„  When  an  error  condition  occurs,  is  it 
passed  to  the  calling  module?  £^1(3), 

4.  Are  the  results  of  a  computation 
cnecked  before  outputting  or  before 


A-l  3 


C  WORKS 
E  CODE/ 


rsrssfjsi 


PORTABILITY 


Is  code  independent  of  word  and 
character  site?  MI. 1(3) 


Number  of  lines  of  machine  language 
statements.  MI.l 
Is  data  representation  machine 
independent?  MI.l (4) 


Is  data  access/storage  system  soft¬ 
ware  independent?  SS.1 


X.  FLEXIBILITY 


1.  Is  module  table  driven  EX. 2(2) 

2.  Are  there  any  limits  to  data 
values  that  can  be  processed? 

G£.2(4) 

3.  Are  there  any  limits  to  amounts 
of  data  that  can  be  processed? 

G£.2(3) 

4.  Are  accuracy,  convergence  and 
timing  attributes  parametric? 

EX. 2(1 ) 


51.  DYNAMIC  MEASUREMENTS  (EFFICIENCY,  RELIABILITY) 


Ouring  execution  are  outputs  within  accuracy  tolerances?  AY. 1(5)  / 

Z .  Qurinq  module/development  testing,  what  was  run  time?  EX. 2(3)  f\ 

7.  Vllr»*+  WAS  W*  +•***  ?  ,  .  f 


/development  testing,  what  was  run  time?  EX. 2(3) 

r\j*s  -rj^<  :  ,  , 


Complete  memory  map  for  execution  of  this  module  SE.1(4) 

Size  (words  of  memory) 

4.  APPLICATION 

<•  SYSTEM 


7.  OTHER 

^  Ouring  execution  how  many  data  Items  were  referenced  but  not  modified  EE. 3(6) 
Ouring  execution  how  many  data  items  were  modified  EE. 3(7) 


XII.  INSPECTORS  COMMENTS _ _ 


.Make  any  general  or  specific  comnents  that  relate  to  the  quality  observed  by  you  while 
applying  this  checklist: 
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WORKSHEET  REPORT 


The  worksheet  report  displays  the  raw  data  entered  In  each  worksheet.  It 
represents  the  current  values  in  the  data  base.  It  is  used  to  verify  and 
track  data  entry. 

AUTOMATED  MEASUREMENT  TOOL 
WORKSHEET  REPORT 
WORKSHEET  3 

DATA  Base  Amtexs 


I.  STRUCTURE  (RELIABILITY,  MAINTAINABILITY,  TESTABILITY) 

1.  NUMBER  OF  LINES  OF  CODE  95. 

2.  NUMBER  OF  LINES  EXCLUDING  COMMENTS  47. 

3.  NUMBER  OF  MACHINE  LEVEL  LANGUAGE  STATEMENTS  0. 

4.  NUMBER  OF  DECLARATIVE  STATEMENTS  4. 

5.  NUMBER  OF  DATA  MANIPULATION  STATEMENTS  5. 

6.  NUMBER  OF  STATEMENT  LABELS  (EXCLUDING  FORMAT  STATEMENTS  0. 

7.  NUMBER  OF  ENTRANCES  INTO  MODULE  1. 

ENTER  [CR]  TO  CONTINUE  '  E'  TO  EXIT: 

8.  NUMBER  OF  EXISTS  FROM  MODULE  2. 

9.  MAXIMUM  NESTING  LEVEL  3. 

10.  NUMBER  OF  DECISION  POINTS  (IF,  WHILE,  REPEAT,  DO,  CASE)  10. 

11.  NUMBER  OF  SUB-DECISION  POINTS  0. 

12.  NUMBER  OF  CONDITIONAL  BRANCHES  (COMPUTED  TO  GO  6. 

13.  NUMBER  OF  UNCONDITIONAL  BRANCHES  (GOTO,  ESCAPE)  0. 

14.  NUMBER  OF  LOOPS  (WHILE,  DO)  4. 

15.  NUMBER  OF  LOOPS  WITH  JUMPS  OUT  OF  LOOPS  0. 

16.  NUMBER  OF  LOOPS  INDICIES  THAT  ARE  MODIFIED  0. 

17.  NUMBER  OF  MODULE  MODIFICATIONS  (SWITH,  ALTER)  0. 

18.  NUMBER  OF  NEGATIVE  OR  COMPLICATED  COMPOUND  BOOLEAN  EXPRESSIONS  0. 

19.  IS  A  STRUCTURED  LANGUAGE  USED?  YES 

20.  IS  FLOW  TOP  TO  BOTTOM  (ABSENSE  OF  BACKWARD  BRANCHING  GOTO's)?  YES 


II.  CONCISENESS  (MAINTAINABILITY) 

1.  NUMBER  OF  OPERATORS  4. 

2.  NUMBER  OF  UNIQUE  OPERATORS  1. 

3.  NUMBER  OF  OPERANDS  8. 

4.  NUMBER  OF  UNIQUE  OPERANDS  3. 

ENTER  (CR)  TO  CONTINUE,  '  E'  TO  EXIT: 
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EXCEPTION  REPORT 


The  exception  report  delivers  the  relationship  of  each  module  to  a  given 
threshold  value  of  a  particular  metric.  The  relationship  (less  than,  equal 
to,  or  greater  then)  and  the  threshold  value  is  input  from  the  user.  This 
report  can  be  used  to  identify  modules  whose  scores  do  not  meet  a  certain 
threshold,  identifying  them  as  potential  problems. 


AUTOMATED  MEASUREMENT  TOOL 
EXCEPTIONS  REPORT 


DATABASE:  AMTEXS  DATE:  12/23/81 

METRIC:  ET.  2 

PHASE:  MODULE  IMPLEMENTATION 
THRESHOLD  VALUE:  0.65 
RELATION:  LESS  THAN 

THE  FOLLOWING  MODULES  ARE  WITHIN  RANGE  REQUESTED 


MODULE  NAME 


VALUE 


NORMALIZATION  REPORT 


The  Normalization  Report  provides  the  user  with  the  overall  rating  of  a 
selected  quality  factor.  A  series  of  regression  equations  are  displayed  which 
have  been  empirically  derived  from  research.  The  current  metric  values  are 
substituted  in  the  equations  and  a  rating  for  the  selected  quality  factor  is 
calculated.  Regression,  equations  exist  for  the  quality  factors  reliability, 
maintainability,  portability,  and  flexibility  only: 

AUTOMATED  MEASUREMENT  TOOL 
NORMALIZATION  FUNCTION  REPORT 


DATABASE:  AMTEXS 

MODULE:  EXSGET  DATE:  12/23/81 


DESIGN  NORMALIZATION  FUNCTION  IMPLEMENTATION  NORMALIZATION  FUNCTION 

FACTOR:  PORTABILITY 

#  NO  DESIGN  NORMALIZATION  FUNCTION  PORTABILITY  =  -1.7  +  .19  (SD.l)  + 

FOR  PORTABILITY  FACTOR  ,76(SD.2)  +  2.5(SD.3)  +  .64(MI.l) 

SD.l  =  0.426 
SD.2  =  0.857 
SD.3  =  1.000 
MI. 1  =  0.972 


PORTABILITY  =  2.154 


METRIC  REPORT 


This  report  calculates  the  value  of  each  metric  catagorized  by  factor  and  by 
development  phase.  This  report  is  used  to  determine  a  total  picture  of  the 
project  as  measurements  are  taken. 


AUTOMATED  MEASUREMENT  TOOL 
METRIC  REPORT /MODULE  IMPLEMENTATION  PHASE 


DATABASE:  AMTEXS 


MODULE:  EXS6ET 

DATE: 

12/23/81 

FACTOR 

CRITERIA 

METRIC 

VALUE 

CORRECTNESS 

Traceability 

TR.l 

1.000 

Completeness 

CP.l 

0.667 

Consi stency/Procedure 

CS.l 

1.000 

Consistency/Data 

CS.2 

0.500 

RELIABILITY 

Consi stency/Procedure 

CS.l 

1.000 

Consistency /Data 

CS.2 

0.500 

Accuracy 

AY.  1 

1.000 

Error  Tolerance/Control 

ET.l 

1.000 

Error  Tolerance/Input  Data 

ET.2 

1.000 

Error  Tol ./Computational  Fail. 

ET.3 

0. 

Design  Structure 

SI.l 

0.625 

Complexity 

SI. 3 

0.100 

Code  Simplicity 

SI. 4 

0.722 

MAINTAINABILITY 

Consi stency/procedure 

CS.l 

1.000 

Consistency/Data 

CS.2 

0.500 

Design  Structure 

SI.l 

0.625 

Complexity 

SI. 3 

0.100 

Code  Simplicity 

SI. 4 

0.722 

Modular  Implementation 

MO.  2 

0.750 

Quantity  of  Comments 

SD.l 

0.426 

Effectiveness  of  Comments 

SD.2 

0.857 

Conciseness 

C0.1 

1.000 

TESTABILITY 

Design  Structure 

SI.l 

0.625 

Complexity 

SI. 3 

0.100 

Code  Simplicity 

SI. 4 

0.722 

Modular  Implementation 

MO.  2 

0.750 

Quantity  of  Comments 

SD.l 

0.426 

Effectiveness  of  Comments 

SD.2 

0.857 

Descriptiveness  of  Impl.  Lang. 

SD.3 

1.000 

PORTABILITY 

Modular  Implementation 

MO.  2 

0.750 

Quantity  of  Comments 

SD.l 

0.426 

FACTOR 


CRITERIA 


METRIC 


VALUE 


: 

Effectiveness  of  Comments 

SD.2 

0.857 

1 1 

Descriptiveness  of  Impl.  Lang. 

SD.3 

1.000 

System  Software/ Independence 

SS.l 

0.500 

Machine  Independence 

MI .  1 

0.972 

m 

REUSABILITY 

Modular  Implementation 

MO. 2 

0.750 

General i ty/ Imp 1 ement  at 1 on 

GE.2 

0.750 

[ 

Quantity  of  Comments 

SD.l 

0.426 

api 

Effectiveness  of  Comments 

SD.2 

0.857 

1 

Descriptiveness  of  Impl.  Lang. 

SD.3 

1.000 

System  Software/ Independence 

SS.l 

0.500 

Machine  Independence 

MI.  1 

0.972 

► 

FLEXIBILITY 

Modular  Implementation 

MO.  2 

0.750 

* 

General Ity/ Implementation 

GE.2 

0.750 

Data  Storage  Expansion 

EX.  1 

0. 

Computational  Extensibility 

EX. 2 

0.500 

Quantity  of  Comments 

SD.l 

0.426 

Effectiveness  of  Comments 

SD.2 

0.857 

Descriptiveness  of  Impl  Lang. 

SD.3 

1.000 

INTEROPERABILITY 

Modular  Implementation 

MO. 2 

0.750 

EFFICIENCY 

Iterative  Processing 

EE. 2 

1.000 

Data  Usage 

EE. 3 

0.668 

A-19 


STATISTICS  REPORT 


The  Statistics  Report  provides  a  profile  of  COBOL  constructs  for  each  module 

AUTOMATED  MEASUREMENT  TOOL 
STATISTICS  REPORT 


DATABASE:  AMTEXS 

MODULE:  EXSGET  DATE:  12/23/81 

NUMBER  OF  LINES  OF  CODE  95. 

NUMBER  OF  PERFORM  STATEMENTS  4. 

NUMBER  OF  EXTERNAL  CALLS  0. 

NUMBER  OF  EXECUTABLE  STATEMENTS  (PROCECURE  DIVISION)  43. 

NUMBER  OF  COMMENTS  48. 

NUMBER  OF  DECLARATIONS  (DATA  DIVISION)  4. 

NUMBER  OF  LABELS  0. 

NUMBER  OF  I/O  REFERENCES  6. 

NUMBER  OF  REDEFINES  (EQUIVALENTS)  0. 

NUMBER  OF  LEVEL  88  DATA  ITEMS  (LOCAL  VARIABLES)  1. 


SUMMARY  REPORT 


The  summary  report  provides  a  summary  of  the  metric  scores  for  all  of  the 
modules  in  the  system. 

AUTOMATED  MEASUREMENT  TOOL 
METRIC  SUMMARY  REPORT 

DATABASE:  AMTEXS  DATE:  12/23/81 

MODULE:  EXSGET 


AY.l  =  1.000 

C0.1  =  1.000 

CP.l 

=  0.667 

CS.l  =  1.000 

CS.2  *  0.500 

EE. 2  =  1.000 

EE. 3 

=  0.668 

ET.l  =  1.000 

ET.2  =  1.000 

ET.3  =  0. 

EX.l 

=  0. 

EX. 2  =  0.500 

GE.2  =  0.750 

MI. 1  =  0.972 

MO.  2 

=  0.750 

SD.l  *  0.426 

SD.2  =  0.857 

SD.3  =  1.000 

SI.l 

=  0.625 

SI. 3  =  0.100 

SI. 4  =  0.722 

SS.l  =  0.500 

TR.l 

=  1.000 

_ 


•  • 


i 
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QUALITY  GROWTH  REPORT 


When  the  user  wishes  to  track  the  value  of  a  particular  metric  over  time,  the 
Quality  Growth  Report  will  furnish  a  tabular  display  of  the  scores  of  a 
selected  metric  over  the  plhases  of  the  project.  This  report  is  used  to  track 
a  particular  metric  through  a  project  to  see  how  its  value  changes. 

AUTOMATED  MEASUREMENT  TOOL 
QUALITY  GROWTH  REPORT 


DATABASE:  AMTEXS 
MODULE:  EXSGET 


DATE:  12/23/81 


METRIC 


DETAILED 

DESIGN 


MODULE 

IMPLEMENTATION 


0.750 


1.000 
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MATRIX  REPORT 


I 

i 


This  report  displays  the  average  and  standard  deviations  for  all  metric  values 
modules.  This  report  displays  all  of  this  information  In  a  matrix  form 
allowing  the  user  to  easily  identify  modules  with  metric  scores  that  vary  from 
the  system  average. 


AUTOMATED  MEASUREMENT  TOOL 
MATRIX  REPORT 


'•  - 


DATABASE:  AMTEXS 

PAGE  *  1 

DATE:  12/23/81 

MODULE  NAME 

AY.l 

CO.  1 

CP.l 

CS.l 

CS.2 

EE. 2 

EXSCEX 

0. 

1.000 

0. 

0. 

0. 

1.000 

EXSCHK 

1.000 

1.000 

0.667 

1.000 

0.500 

1.000 

EXSCLP 

1.000 

1.000 

0.667 

1.000 

0.500 

1.000 

EXSDBG 

0. 

1.000 

0. 

0. 

0. 

0. 

EXSGET 

1.000 

1.000 

0.667 

1.000 

0.500 

1  000 

EXSHLP 

0. 

1.000 

0.833 

1.000 

0.500 

0. 

EXSPGR 

0. 

1.000 

1.000 

1.000 

0.500 

1.000 

EXSQRY 

0. 

1.000 

0.667 

1.000 

0.500 

0. 

EXSSSM 

0. 

1.000 

1.000 

1.000 

0.500 

0. 

EXSUPK 

0. 

1.000 

0.625 

1.000 

0.500 

1.000 

AVERAGE  = 

0.300 

0.900 

0.550 

0.700 

0.350 

0.500 

STANDARD  DEVIATION  = 

0.438 

0.316 

0.401 

0.483 

0.242 

0.527 
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MODULE  REPORT 


This  report  displays  the  catalog  of  modules  that  have  been  entered  Into  the 
database.  It  provldss  a  status  report  on  the  database. 

AUTOMATED  MEASUREMENT  TOOL 
MODULES  REPORT 

DATABASE:  AMTEXS  DATE:  12/23/81 

WS1  CONTAINS  SOME  NIL  VALUES 

WS2A  CONTAINS  SOME  NIL  VALUES 

THE  FOLLOWING  MODULES  ARE  PRESENTLY  IN  THE  CURRENT  DATABASE: 

1.  EXSCEX  **  2.  EXSCHK  * 

3.  EXSCLP  *  4.  EXSDBG  ** 

5.  EXSGET  *  6.  EXSHLP  * 

7.  EXSPGR  *  8.  EXSQRY  * 

9.  EXSSSM  *  10.  EXSUPK  * 

TOTAL  NUMBER  OF  MODULES  IN  DATABASE  IS  10. 

NOTE:  *  INDICATES  BOTH  WS2B  AND  WS3  CONTAIN  SOME  NIL  VALUES. 

NOTE:  **  INDICATES  WS2B  CONTAINS  SOME  NIL  VALUES. 


SECTION  B-l 
INTRODUCTION 


When  designing  the  AMT  (Automated  Measurement  Tool),  the  portability  of  the 
IFTRAN  source  code  was  a  major  consideration.  The  AMT  contract  stipulated 
that  a  fully  running  version  of  the  AMT  be  delivered  on  the  Honeywell  6000 
series  computer  (GCOS  operating  system)  located  at  RADC  (the  Rome  Air 
Development  Center,  Griff iss  Air  Force  Base,  New  York).  In  order  to  provide 
us  with  more  efficient  computer  access,  it  was  decided  to  develop  the  software 
on  a  VAX  11/780  computer,  at  our  General  Electric  Facility  in  Sunnyvale, 
California  and  then  ship  a  tape  containing  the  source  code  to  RADC,  Therefore, 
the  VAX  version  had  to  be  implemented  in  very  standard  code  using  system 
dependent  functions  only  when  absolutely  necessary.  Whichever  system 
dependent  functions  were  used  would  be  modified  after  the  code  had  been  moved 
to  the  Honeywell.  This  appendix  describes  the  coding  techniques  used  to 
assure  the  AMT  code  was  kept  as  system  independent  as  possible  and  the 
differences  between  the  VAX  and  Honeywell  system  dependent  functions. 


SECTION  8-2 
CODING  STANDARDS 


Table  82-1  describes  the  standards  established.  Table  B2-2  Identifies  the 
differences  In  the  system  dependent  functions  between  the  two  computer 
environments. 


Table  B2-1  Code  Standardization 


1. -  Only  INTEGER  and  REAL  data  types  used. 

2.  No  INTEGER*n  data  types  used. 

3.  No  LOGICAL  data  types  used. 

4.  No  CHARACTER  or  CHARACTERS  data  types  used. 

5.  Character  strings  are  stored  In  Integer  arrays,  one  character  per 
array  element.  Unused  array  elements  are  filled  with  blanks. 

6.  Input/Output  of  "character"  arrays  Is  performed  with  the  implied  DO  in 
combination  with  the  alphanumeric  (A)  field  Format  descriptor. 

Example:  WRITE( CRT. 100) (DBNAME( I) ,  1=1,  15) 

100  FORMAT  (  'DATABASE  NAME'  ,  15A1) 

7.  System  dependent  functions  (mainly  file  handling  functions)  are 
isolated  Into  subroutines.  This  way,  modifications  to  those  functions 
need  only  be  made  In  one  place. 


B-3 


Table  B2-2  System  Dependent  Function  Differences 


-~w 


1.  Creating  Files 
VAX 

OPEN  (UNIT  *  n,  NAME  «  filename,  TYPE  =  'NEW') 

VAX  FORTRAN  allows  filename  to  be  an  integer  array,  which  is  the 
way  AMT  stores  character  strings. 


H6000 

CALL  CALLSS  ("ACCESS  CF,  filename,  size,  type") 

or 

CALL  CALLSS  (  string  ) 

where  string  *  "ACCESS  CF,  filename,  size,  type" 

Honeywell  FORTRAN  has  no  direct  method  for  creating  files.  The  CALLSS 
routine  allows  arr^  tlmeshare  command  to  be  given  from  an  executing  FORTRAN 
program  (the  tlmeshare  command  being  a  character  string  enclosed  in 
quotes).  In  this  case,  the  ACCESS  subsystem  is  called  to  create  a  new 
file.  Note  that  AMT  stores  filenames  in  integer  arrays.  Therefore,  in 
order  to  call  CALLSS,  each  character  stored  in  the  filename  integer  array 
must  first  be  concatenated  into  the  timeshare  command  character  string. 
Concatenation  is  performed  by  the  Honeywell  CONCAT  routine. 

2.  Opening  Files 
VAX 

OPEN  (UNIT  *  n,  NAME  *  filename) 


-  r 
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Table  B2-2  System  Dependent  Function  Differences  (Cont.) 

H6000 

CALL  ATTACH  (unit,  "filename;",  etc.) 

Note  that  filename  must  be  a  character  string.  Each  character 
stored  in  an  AMT  filename  Integer  array  must  first  be 
concatenated  into  the  filename  character  string.  Also  note  that 
the  filename  character  string  must  be  terminated  by  a  semicolon. 

3.  Closing  Files 
VAX 

CLOSE  (UNIT  =  n,  DISPOSE  =  'SAVE') 

H6000 

CALL  DETACH  (unit,  etc.) 

4.  Determing  If  a  File  Currently  Exists 
VAX 

CALL  LOOK  (unit,  filename,  blocks,  return  code) 

After  calling  LOOK: 

IF  return  code  *  0,  the  file  exists. 

IF  return  code  *  1,  the  file  exists,  but  is  currently  open. 

Any  other  return  code,  the  file  does  not  exist. 

H6000 

CALL  ATTACH  (unit,  filename,  status,  etc.) 

(See  Opening  Files) 

After  calling  ATTACH: 

If  status  *  octal  (4000  0000  0000)  or 
octal  (4004  0000  0000) 

the  file  exists. 

IF  status  *  octal  (4037  0000  0000) 


Table  82-2  System  Dependent  Function  Differences  (Cont.) 


the  file  exists,  but  Is  Currently  open. 

Any  other  status 

the  file  does  not  exist. 

5.  Opening  Random  Access  Files 
VAX 

OPEN  normally 

H6000 

Before  accessing  random  files  a  call  to  RANSIZ  must  be  made  to 
specify  the  record  size  of  the  file.  The  file  Is  then  opened 
normally. 

6.  Suppressing  Carriage  Return  and  Line  Feed 
VAX 

End  FORMAT  statement  with  dollar  signfleld  descriptor.  The  cursor 
will  remain  positioned  at  Its  current  location  for  the  next  write. 
FORMAT  (5X,  15A1,  $) 


H6000 

Place  an  ampersand  as  the  first  print  character  In  FORMAT  statement. 
The  write  will  begin  where  the  cursor  was  previously  positioned. 
FORMAT  (  F6.2) 

7.  PROGRAM  Statement 
VAX 

Allows  the  PROGRAM  statement  as  the  first  line  of  a  FORTRAN  program. 

H6000 

Does  not  recognize  the  PROGRAM  statement. 
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SECTION  B-3 

VAX  TO  H6000  TRANSFER  TAPE 


Copying  Files  from  the  VAX  to  the  Transfer  Tape  (see  also  Section  B-4). 

1.  Create  a  file  named  TRANSFER. LST.  Enter  into  TRANSFER. 1ST  the  name 
of  each  file  to  be  transferred  (one  filename  per  line). 

2.  $  ALLOCATE  MTAn: 

3.  Physically  mount  tape  on  drive  #n. 

4.  $  INITIAL IZE/DENSITY=1 600  MTAn:  label 

5.  $  MOUNT /DENS ITY=1 600 /FORE IGN/BL0CK=80  MTAn:  label 

6.  $  RUN  TAPE2 

When  prompted  for  the  output  file  name  enter  MTAn: 

(where  n  *  number  of  tape  drive  allocated) 

When  prompted  for  the  VAX  file  list  enter  TRANSFER. LST 

At  the  end  of  its  execution  TAPE2  will  display  the  message  ALL 

FILES  COPIED 

7.  $  DISMOUNT  MTAn: 

8.  Physically  remove  the  tape. 

9.  $  DEALLOCATE  MTAn: 

The  tape  is  now  ready  to  be  sent  to  RADC. 

Reading  the  VAX  Tape  on  RADC' s  H6000  (see  also  Section  B-6) 

1.  Obtain  the  RADC  tape  number  assigned  to  the  transfer  tape. 

2.  Edit  file  /AMTS/TRANSLATE /TMP ,  substituting  the  new  tape  number  in 
the  TAPE9  IN  card. 
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3.  Create  a  sequential  file  named  /AMTS/VAXDATA  of  maximum  size  1000. 
The  transfer  tape  will  be  written  to  this  file. 

4.  * CARO IN 

5.  *RUN  /AMTS/TRANSLATE/TMP 

6.  When  the  job  has  finished  running: 

♦CONVERT  /AMTS/VAXDATA  =  * : TRAIL 


7.  *R£SAVE  /AMTS/VAXDATA 

/AMTS/VAXDATA  now  contains  the  information  that  was  stored  on 
the  transfer  tape. 


SECTION  B-4 
VAX  LISTINGS 

1.  Listing  of  VAX  File  TRANSFER. LST. 

100  EXSCEX.IFT 
200  EXSDBG.IFT 
300  RGSCMM.IFT 
400  UTLCRE.IFT 
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APPENDIX  C 
DBMS  SURVEY 
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SECTION  C-l 
PURPOSE 


One  of  the  reouirements  of  the  contract  was  to  attempt  to  utilize  a  DBMS  in 
our  system  design.  It  was  anticipated  that  the  use  of  a  DBMS  would  Increase 
the  portability  of  the  system  as  well  as  reduce  the  required  effort  to  develop 
the  system.  Because  these  assumptions  did  not  hold  true  for  MDQS,  the  DBMS  on 
the  H6000/GC0S  target  environment,  a  OBMS  survey  was  conducted. 

Section  2  discusses  the  problems  of  using  MDQS.  It  states  why  the  initial 
version  of  the  AMT  will  have  to  be  implemented  with  its  own  built-in  data 
management  functions. 

Section  3  examines  the  utility  of  using  alternative  DBMS's  deemed  most  likely 
to  be  available  in  target  environments.  These  DBMS's  were  examined  and 
compared  according  to  certain  criteria. 


SECTION  C-2 

THE  PROBLEMS  OF  USING  MDQS 


The  use  of  a  DBMS  was  initially  considered  to  be  a  good  design  choice. 
Storage  and  retrieval  of  the  metric  data  could  be  performed  by  the  DBMS.  It 
has  turned  out  that  the  Honeywell  provided  DBMS,  MDQS,  Is  inappropriate  for 
the  AMT  application.  Thus,  while  DBMS's  In  general  could  be  considered  for 
other  versions  of  AMT,  the  initial  version  on  the  RADC  H6180/GC0S  system  will 
have  to  be  implemented  with  its  own  built-in  data  management  functions. 

The  two  major  reasons  for  not  using  MDQS  are:  (1)  we  would  not  be  able  to  use 
any  of  our  existing  software,  and  (2)  the  system  would  not  be  transportable. 
MDQS  is  a  completely  self-contained  DBMS.  It  was  developed  strictly  for 
business  applications  in  which  data  Is  simply  stored  and  retrieved  with  a 
minor  amount  of  manipulation.  The  manipulation  Is  done  by  Internal  procedures 
written  using  MDQS  -  provided  constructs.  Thus,  all  of  AMT  would  have  to  be 
implemented  within  the  framework  of  MDQS.  This  is  practically  Impossible 
considering  the  complexity  of  the  parsing  and  measuring  algorithms  that  are 
part  of  AMT.  In  addition,  none  of  the  existing  code  that  performs  the  parsing 
function  (approximately  2000  lines  of  code)  could  be  used. 

More  importantly,  if  the  AMT  was  developed  under  the  framework  of  MDQS  It 
would  have  to  be  totally  converted.  This  conversion  would  be  necessary  either 
for  interfacing  with  another  DBMS  of  for  running  on  another  system. 
Constralnment  of  the  portability  of  this  prototype  software  development  Is 
unacceptable.  The  trade-offs  of  using  or  not  using  MDQS  are  summarized  below: 

Using  MDQS:  Advantages 

o  Query  capability 
o  Data  management  routines  provided 
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Disadvantages 

•  Resulting  system  not  portable  to  370  or  11/70 

•  No  existing  software  could  be  used. 

Because  of  the  net  unf avorabll  1  ty  of  using  HDQS,  our  approach  has  been  to 
develop  some  very  basic  data  management  functions  based  on  a  data  base 
specifications.  These  data  management  function  provide  the  core  functions  of 
a  DBMS.  The  system  dependencies  are  isolated  in  a  few  of  these  routines. 
They  will  have  to  be  re-written  when  the  system  is  transported  to  another 
system.  This  is  a  significant  improvement  in  the  degree  of  portability  of  th< 
system. 


SECTION  C-3 

ALTERNATIVE  OBHS' S  FOR  CONSIDERATION  IN  FUTURE  AMT  VERSIONS 

In  order  to  examine  the  utility  of  alternative  DBMS's,  we  did  a  survey.  Table 
C3-1  gives  an  overview  of  the  DBMS's  we  considered  which  have  the  facilities 
to  run  on  the  target  environments  hardware  and  operating  systems.  Tables  C3-2 
through  C3-7  Include  a  detailed  analysis  of  only  those  DBMS's  thought  likely 
to  best  fit  the  target  environments  on  overall  criteria.  MDQS  is  also 
included.  Of  these  latter  DBMS's  only  the  following  are  capable  of  being 
called  from  FORTRAN: 

•  TOTAL 

•  IDMS 

•  MRDS 

•  MR  I 


Accordingly,  selection  from  this  subset  of  four  DBMS's  would  contribute  the 
most  portabiltiy  to  future  AMT  versions  in  the  target  environments,  all  other 
things  being  equal. 


Table  C3-1 

Data  Base  Management  Systems 


I.  I8M  370/OS 

A.  Self-Contained  Systems 

1.  ARAP-Data  Retrieval  System 

(IBM  370/115  and  up,  0S/VS1 ,  0S/VS2.  Interfaces  with 
OS  Telecomm  subroutines.) 

2.  Computer  Corporation  of  America  -  Model  204 
(IBM  370  under  OS/MFT,  OS/MVT). 

3.  Infodata  Systems  Inc.  -  INQUIRE 
(IBM  370  interfaces  with  OS) 

4.  Mathematica  Inc.  -  RAMIS 

(IBM  370  under  OS.  Uses  OS  facilities  for  I/O,  but 
relies  on  no  other  system.  Dependent  utlities.  Dependent 
utilities,  contains  own  sort  logarithm.  TP  and  Timesharing 
interfaces  are  available). 

5.  Meade  Technology  Corp.  -  DATA/CENTRAL 

(IBM  370,  Model  40/135  up.  Operates  under  all  versions  of 
OS  including  virtual.  Implementation  on  new  machine  re¬ 
quires  12-18  months). 

6.  MR I  Systems  Corporation  -  SYSTEM  2000 
(IBM  370  OS  VM/CMS ,  OS/llOO). 

7.  National  CSS-  NOMAD 
(IBM  370  OS) 

8.  TRW  OIM  II 
(IBM  370  OS/VS) 

B.  CODASYL  -  Type  Systems 

1.  Cull  inane  Corporation  -  IDMS 
(IBM  370,  all  operating  systems) 

2.  International  Data  Base  Systems  -  SEED 

(Written  in  FORTRAN  so  it  may  be  used  on  any  machine 
with  a  FORTRAN  computer.  CPU's  include  IBM  370). 

C.  HOL  -  Based  Non  -  CODASYL  SYSTEMS 


1.  Cincom  Systems,  Inc.  -  TOTAL 
(IBM  370  OS) 


2.  IBM  Corporations  -  IMS 


(IBM/VS  runs  on  System  370  models  138,  145,  148,  15511,  158, 
16511,  168  and  3033,  0S/VS1  and  0S/VS2). 

3.  Insyte  Oata  Corp.  -  DATA  COM/DB 
(IBM  370,  OS) 

4.  Software  Ag  -  ADABAS 
(IBM  370,  OS  MVT) 

II.  Honeywell  6180/GC0S 

A.  Self  -  Contained  Systems 
1 .  MOQS 

B.  CODAS YL  -  TYPE  Systems 

1.  Honeywell  information  Systems  IDS  II 

(Honeywell  L6,  L64,  and  6000/L66  systems  operating  under  6C0S 
batch  or  communications  environment). 

C.  HOL  -  Based  NON  -  CODASYL  Systems 
1.  Cincom  Systems,  Inc.  TOTAL 

(Honeywell  Level  62  GCOS;  Level  66/6000  GCOS) 

III.  Honeywell  6180/MULTICS 
A.  Self  -  Contained 

1.  Honeywell  Multics  Relational  Data  Store  -  MRDS 
(HIS  Series  60/Level  68  Hardware). 

IV.  PDP  11/70  UNIX 


A.  Self  -  Contained  Systems 

1.  Bell  LABS  -  INGRESS 
(PDP  11/70;  UNIX) 

B.  CODSDYL  -  TYPE  Systems 

1.  Cull  inane  Corporation  -  IDMS  -11 
(similar  to  IDMS;  see  IDMS  chart) 


FUNCTION 


PROPERTY 


PARAMETERS 


I.  DATA  BASE 
DEFINITION 


A.  Item 

Description 

Self-contained  DDL.  Items  defined  in  terns  of  unique 
name  and  number.  Names  up  to  250  characters  in  length. 
Data  Types:  integer,  decimal,  cnaracter,  text,  data, 
money.  Variable  lengtn  character  and  text  fielas  uo 
to  250  characters  long.  Any  number  of  user-soeci fled 
indexed  fields. 

3.  Logical 

Structure 

1 

Tree  structure.  Data  Base  Definition  allows  32  levels 
of  Schema  records  containing  schema  items  to  model 
entries  of  data  records  containing  data  items.  Oata 
Bases  may  be  linked  using  HOL  Interface  to  form  logical 
networks. 

i  C.  Physical 
j  Structure 

Each  data  base  is  a  stand-alone  entity,  comprised  of 
separate  physical  files  for  schema,  data,  structures, 
and  indexes.  Data  stored  In  user-oetermined  fixed- 
length  blocks  in  physical  hierarcnies  system  does 
own  deblocking.  File  inversion  on  selected  fields. 

D.  Access 

Methods 

80AM,  BSAM,  BP  AM,  QSAM. 

E.  Special  Storage  j  Alphanumeric  fields  are  variable  length  within 
Techniques  fixed  length  logical  record  via  separate  Extended 

Field  Table.  Hierarchical  structure  is  used  to 
j  eliminate  data  redundancy.  A  variety  of  storage 
|  techniques  (ring,  tree,  dense  list,  etc.)  are  used 
|  to  optimize  specific  DBMS  processes. 


II.  OATA  BASE 
CREATION 
ANO 

HI  REVISION  j 

I 


Initial  data  load  may  be  run  as  a  one  time,  incre¬ 
mental  or  transaction  processing  procedure.  This 
loading  can  be  accomplished  with  any  combination 
of  Programming  Language  Extension, ^Self-Containe 
Language  and/or  Self-Contained  Utility. 


Schema  may  be  modified  using  sel f-cantaineo 
language.  System  automatically  performs  any  internal 
restructuring  required. 


VI.  DATA  '  A.  Selection 

MANIPULATION)  Level 

, - - 

!  a.  Operators, 

:  Comparators, 

;  Logical 

Complexity 


Selection  is  at  the  item  level.  Items  may  be 
identified  by  schema  name  or  alias. 


Variety  of  CML'S:  PLEX,  SCL,  RW.  Ccmprenensive 
selection  capability  in  all  DML's.  Uses  Boolean, 
Threshold,  indexed/non-indexed,  text  search  and 
positional  techniques.  Local  and  global  updates 
with  dynamic  reuse  bf  deleted  space.  SCL  supports 
virtual  data  items. 


X 
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FUNCTION  j 

PROPERTY 

parameters 

C.  Reporting 

1 

| 

Self  Contained  Languages  suitable  for  heuristic 
browsing,  single  reporting,  ano  complex,  formal 
report  generation  are  available.  9oth  user* 
defined  and  formal  default  ootions  are  provided. 

Report  requests  may  be  formulated  dynamically  or 
executed  from  data  base  stored  procedures.  Sorts, 
j  arithmetic  expression  processing,  logic  structures, 

1  and  system  supplied  function  are  also  included. 

V.  USER 

INTERFACE 

A.  Manipulation 
Languages 

|  English-like.  Self-contained  Query/Update  lan- 
!  guage  for  single/multiple  user  interactive  and 

batch  processing. 


: 

r  ■* 

B.  Mode  of 

Interaction  ! 

i 

Interactive  or  batch.  All  Self-Contained  and 

HOL  languages  supported  in  both  batch  and  interactive 
modes.  Interactive  supoort  via  MRI's  TP  2000, 

CICS,  INTERCOM,  TSO,  and  others. 

i 

C.  Error 

Messages 

English-like  text  messages  provided  for  self- 
contained  language  user,  and  diagnostic  return 
codes  and  messages  for  host  language  programs. 
Centralized  Messages  and  Codes  Manual.  Microfiche 
early  warning  for  systems  support  personnel. 

0.  Documentation  j 

1 

1 

Modular  documentation  designed  to  satisfy  the 
needs  of  each  type  of  user.  Structured  top-down 
from  concepts  to  language  specifications  to  admin¬ 
istration  and  supoort.  Strong  reliance  on  examples 
and  usage  guidelines. 

VI.  application 
PROGRAMMING 

A.  HOL  Interface  ! 

i 

1 

1 

i 

1 

Interface  available  for  assembly,  COBOL,  FORTRAN, 
and  PL/1  languages,  HOL  interface  includes  a 
precompiler  which  transforms  English-like 
comnands  embedded  in  the  HOL  cade  ta  DBMS  call 
connands  and  the  necessary  parameter  lists.  The 
interface  allows  run-time  HOL  interaction  with  ua 
to  sixteen  open  data  bases  at  any  point  in  time. 

3.  Subroutine 
Capabilities 

i 

j 

I 

J 

i  1 

1 

I 

i 

i 

i 

» 

i 

Modular  HOL  programming  is  supported  with  DBMS 
processing  available  to  main  line  and  suborqinac^ 
modules.  SCL  commands  ('strings")  may  be  store- 
with  the  data  base  definition  and  executed  by 
entering  the  string  name.  Parameters  may  be  passes 
|  at  execution  time.  Strings  may  be  called  by  other 
!  strings  and  Doth  retrievals  and  uodate  may  be 
|  performed.  External  conmand  files  may  be  read. 

!  Calculation  definitions  may  be  storsc  as  virtual 
j  items  in  the  cata  base.  Calculation  definitions 
j  may  be  parametric. 

i 

1 

C-9 


- 


Table -C3-2 
MR I  (cont.) 


FUNCTION 

PROPERTY  !  PARAMETERS 

1 

l 

C.  Special 

Operators 

j 

!  HOL  support  for  dynamic  subsets  (LOCATE),  network 
retrievals  (LINK),  sorts  (ORDER  3Y),  and  automatic 

1  return  code  processing  (FOR  RC)  are  examples  of  spe- 
I  cial  operators. 

1  Special  operators  in  the  SCL  includes  histoarams 
system  functions  (SUM,  AVERAGE,  STD  DEVIATION, 

COUNT,  MINIMUM,  MAXIMUM),  and  user  defined  calcu¬ 
lations  using  the  ()■*■-*  /  symbols  are  examples. 

I 

D.  I/O  Outside 

OMS 

I 

Any  format  output  file  from  HOL  Interface  Program. 

Report  files  created  by  Self-Contained  Language 
;  and  Report  Writer.  Unload  to  value  string  format 
i  provides  capability  to  move  data-base  across 
!  hardware  types. 

E.  Auxiliary  j  Intermediate  results  may  be  stored  and  maniDulated 

Storage  :  in  the  self-contained  report  writer.  Work  areas, 

database  table  pages,  sort  and  scratch  files  managed 

1  by  DBA  tuneable  Buffer  Manager. 

VII.  DATA  BASE 
SECURITY, 
INTEGRITY 
AND 

ADMINIS¬ 

TRATION 

A.  Data 

Validation 

Automatic  checking  on  all  fields  cased  on  data  type. 
Further  data  checking  may  be  performed  by  user  HOL 
programs.  Customized  data  validation  via  the  user 
exit  facility  of  the  Universal  Software  Interface. 

3.  File 

Protection 

Security  via  passwords  at  item  level.  Authorization 
for  retrieval,  update  and/or  qualification  optional. 
Security  by  data  value  at  the  hierarchical  record 
level.  User  exits  available  for  custom  security 
checking. 

C.  Sur/ell  lance 

Two  levels  of  logging  (accounting  and  system  usage) 

[  provide  data  suitable  for  surveillance  needs. 

0.  Failure 

Protection 

1 

i 

! 

! 

, 

i 

Multiple  recovery  techniques  which  allow  the  DBA  to 
trade  logging  overhead  for  recovery  speed.  Capa¬ 
bilities  range  from  automatic  rollback  to  self- 
contained  dump/restore  utilities.  Recovery  tech¬ 
niques  can  be  specified  for  each  Individual  da??,  base 
and  can  be  chanaed  upon  command. 

1 
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IBM  37G/QS 

I  QMS:  CODASYL-TYPE  DATA  BASE  MANAGEMENT  SYST 


FUNCTION 


I.  DATA  BASE 
DEFINITION 


i 


PROPERTY 


A.  Item 

Description 


B.  Logical 
Structure 


C.  Physical 
Structure 


PARAMETERS 


User  -  assigned  names.  Formats  are  those  of  sup¬ 
porting  host  language. 


Network  structure  is  through  CODASYL'S  set  relation 
ships.  Several  types  of  relationship  possible. Mem¬ 
bership  may  be  mandatory  or  optional,  manual  or 
automatic.  Linkage  options  allow  one  or  bidirec¬ 
tional  pointer  chains  and  member  to  owner  pointers. 


User  may  specify  storage  area  for  occurrences  of 
record  type.  Other  options:  System  handles  all¬ 
ocation  and  optimization  of  peripheral  storage. 
OB  administrators  may  assign  DB  portions  to 
physical  areas. 


E.  Access  |  Chained,  direct,  randomized,  sequential,  sec 

Methods  i  ondary  index. 

I 

F.  Special  Storage (  Space  management  paging  technique. 

Techniques  j 


P  DATA  BASE  ; 

(  CREATION  | 

AND  ' 

III  REVISION  | 

I 

i 


IV.  DATA  |  A. 

MANIPULATION  j 


i 


C. 


V.  USER  ;  A. 

INTERFACE 


;  B. 


j  Input  via  user  application  programs,  or  load 
j  utility. 

j  Modification  of  total  DB  descriptions  (schema) 
can  be  handled  either  with  a  reload  or  restruc- 
I  ture.  Subschema  can  be  modified  at  any  time. 


Selection  j  At  record  level  by  record  identifier  or  placement 

Level  I  relative  to  other  records,  or  secondary  indexes. 


Operators, 

Comparators, 

Logical 

Comolexity 


Reporting 


Function  of  the  host  language  and  user  programs. 


i  Reporting  done  through  user  program.  OLQ 
facility. 


Manipulation 

Language 


COBOL,  PLI,  Assembler  Macio,  CALL,  FORTRAN 


Mode  of  CV  option  allows  several  DMS  tasks  to  share  same 

Interaction  copy  of  system  in  a  multitask  environment.  CV 

perfoms  task  monitoring  and  threading  of  DBMS 
calls.  System  includes  monitor  interface  and  a 
TP  monitor.  With  monitor,  each  task  can  access 
any  DB  areas  available  for  the  user's  declared 
:  usage  mode.  GCI  ensures  that  more  than  one  task 
|  does  not  update  the  same  record  a  ‘he  same  time, 
i  Mul ti -threading  and  multi-tasking  central  version. 
Intergrated  DB/DC  functions. 


N 


& 


i 


i 


1 

1 

c. 

i 

Error  j 

Messages  i 

1 

Compilation  errors  listed  with  COBOL  source 
statements.  Error  status  is  returned  after  DML 
statement  execution  for  all  host  languages. 

0. 

Documentation 

Standard  user  documentation. 

VI. 

application 

PROGRAMMING 

A. 

Hoi 

Interface  1 

i 

Through  call  statements. 

1 

B. 

Subroutine 

Capabilities 

Function  of  user's  application  program. 

i 

c. 

f 

Special 

Operations 

Limited  to  those  provided  by  host  language. 

D. 

I 

i 

I/O 

Outside  OMS 

Done  through  user  programs. 

!  E. 

j 

Auxiliary 

Storage 

!  To  be  provided  by  user  on  his  program  area. 

VII 

DATA  BASE 
SECURITY, 
INTEGRITY 
& 

ADMINIS¬ 

TRATION 

A. 

i 

Data 

Val idation 

| 

Data  item  integrity  is  user's  responsibility. 

Record  placement  is  verified  by  program  following 
user  placement  options.  System  provides  data 
dictionary  reports  to  DB  administrator  to  docu- 
j  ment  DB  contents. 

:  B. 

| 

1 

» 

! 

File 

Protection 

Access  restrictions  via  subschema.  Normal, 
protected  or  exclusive  retrieval  or  update  can  be 
specified  for  each  area.  Record  level  lock  for 
concurrent  update  and  deadlock  protection. 

;  c. 

i 

Surveillance 

' 

[ 

1  Security  dump  provides  DB  copy  and  statistics 
of  DB  contents.  Any  part  of  dump  may  be  re¬ 
loaded  using  the  security  restore  utility. 

!  D- 

Failure 

Protection 

j  Restart/recovery  utilities 

i 

i 

I 

I 


i 


i 
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HONEYkElL  6180/GCOi 

TOTAL:  HOL-BASED  DATA  BASE  MANAGEMENT  SYSTEM 


FUNCTION 

PROPERTY 

PARAMETERS 

I. 

DATA  BASE 
DEFINITION 

A. 

Item 

Description 

User  assigned  names.  Data  formats  are  those  of 
supporting  host  language. 

B. 

Logical 

Structure 

Network  multilist  structure  implemented  via  chains 
of  bidirectional  pointers  linking  variable  entry 
records  on  the  basis  of  relationships  specified  by 
user.  DB  elements  include  items,  groups,  records, 
files.  Multiple  linkage  paths  may  be  extended  over 
several  data  bases.  Each  linkage  path  corresponds  to 
a  single  entry  file  which  provides  pointers  to  the 
chains  of  the  linkage  path. 

C. 

Physical 

Structure 

Records  are  fixed  length,  although  several  record 
formats  on  any  given  file.  Single-entry  records  are 
accessed  by  a  randomizing  procedure  using  a  key  value. 
Variable  entry  records  are  then  accessed  following 
pointer  chains.  All  data  sets  can  also  be  accessed 
serially. 

Access 

Methods 

Disk  access  is  through  BDAM  and/or  VSAM. 

e. 

1 

Special  Storage 
Techniques 

Multiple  files  can  share  an  I/O  buffer  as  specified 
by  user,  but  single  and  variable  entry  data  sets  may 
not  share  same  I/O  buffer.  A  linkage  path  may  be 
specified  as  "primary"  to  optimize  physical  placement 
of  records.  TOTAL  provides  dynamic  reallocation  of 
space  and  optimization  of  synonym  chains  as  well  as 
user  control  parameters  which  optimize  seek  time. 

II. 

DATA  BASE 
CREATION 

! 

! 

Input  via  user  application  programs  or  optional 
database  administrator  utilities. 

Ill 

DATA  BASE 
REVISION 

1 

i 

! 

i 

} 

New  records  can  be  added,  deleted,  or  modified  from 
existing  files.  New  data  sets,  linkage  paths,  record 
elements  and  modification  of  storage  areas  require 

DB  regeneration,  but  not  necessarily  program  or 

DB  file  modification. 

IV. 

DATA 

MANIPULATION 

A. 

i 

Selection 

Level 

At  field  level  based  on  field  values.  Items  described 
by  name  or  by  position  (in  the  case  of  records)  along 
with  the  linkage  path. 

B. 

Operations, 
Comparators , 

Logical 

Complexi  ty 

Standard  comparators.  Complexity  is  function  of 
user  program. 

;  C. 

! 

Reporting 

Reporting  via  user  programs  with  optional  on-line 
query  and  batch  reporting  system  capability. 

Output  to  all  devices  available  to  user  programs. 


C-l  3 


B 


1 


.  jt i e  Co--  Ccr: .  I 
HONEYWELL  61SC/GCCS 


TOTAL:  HOL-BASED  DATA  BASE  MANAGEMENT  SYSTEM  (cont.) 


FUNCTION  !' 

PROPERTY  : 

1 

PARAMETERS 

V.  USER 

INTERFACE 

! 

A.  Manipulation 
Language 

Any  language  supporting  subroutine  calls. 

1 

j 

B.  Mode  of 

Interaction  I 

! 

TOTAL:  Batch  or  on-line  full  multi-task,  multi¬ 
thread  system.  Transaction  logging  (before  and  after 
images).  Records  locked  (one  per  file  per  task)  when 
task  is  updating.  If  locked  record  is  requested  by 
other  tasks  (as  monitored  by  the  system)  it  is  released 
on  a  "time/request"  algorithm  to  other  tasks.  Original 
task  will  be  posted  with  a  status  indication. 

C.  Error 

Messages  i 

I 

Error  condition  returned  through  user  specified 
status  variable  when  using  data  manipulation 
language. 

D.  Documentation 

j 

i 

Total  DBA,  total  applications  reference  manual,  total 
utilities,  batch  retrieval  user  guide,  comprehensive 
retrieval  user  guide,  on-line  query  user  guide, 
data  dictionary  manual,  data  directory  manual,  on-line 
directory  maintenance  manual. 

APPLICATION 

PROGRAMMING 

i 

j  A.  HOL  Interface 

Oata  manipulation  is  via  user  application  programs 
that  issue  calls  to  TOTAL. 

1 

|  B.  Subroutine 
i  Capabilities 

!  Function  of  the  supporting  host  language. 

i 

j  C.  Special 

J  Operations 

j  Function  of  the  supporting  host  language. 

1 

j  D.  I/O  Outside 

DMS 

1 

t 

j  Done  through  user-provided  programs. 

E.  Auxiliary 

Storage 

1 

1  To  be  provided  by  user  on  h’s  program  area. 

j 

VII.  DATA  BASE 

A.  Data 

Structure  validity  provided  by  system.  Additional 

SECURITY 

INTEGRITY, 

& 

Validation 

integrity  checking  obtainable  via  special  system 
exit  to  DB  administrator  programs. 

ADMINIS¬ 
TRATION  B.  File 

Protection 


Special  exit  is  provided  for  interface  with  user-provided 
security  procedures.  Full  DBA  capabilities  to  control 
user  access  include  sub-schema  (lo  ical  view)  which 
specifies  user  password,  usable  se*  f  elements  (data 
item  names)  and  inter/intra  file  access. 
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TOTAL:  HOL-BASED  DATA  BASE  MANAGEMENT  SYSTEM  (cont.) 


FUNCTION 

PROPERTY 

PARAMETERS 

■ 

Content  validity  must  be  assured  by  :;ser  application 
program. 

D.  Failure 

Protection 

i 

Restart/recovery  procedures  are  provided.  Forward 
and  backward  processing  of  update  history,  optional 
automatic  task  level  checkpoint,  and  other  capabilities 
available  under  ENVIRON/1  (Cincom  TP  monitor),  and 

CICS. 

■n 


•2D:c  C3“£ 
i £ V.  370/CS 


IMS:  HOL-BASED  NON- 

■CODASYL  DATA  BASE  MANAGEMENT  SYSTEM 

function 

PROPERTY 

PARAMETERS 

I.  DATA  BASE 
DEFINITION 

: 

A.  Item 

Description 

1 

Items  described  in  data  base  description  (DBD)  for  DBMS 
sequencing  and  selection  function,  described  in  program 
at  segment  level  for  standard  program  use.  No  restric¬ 
tions  on  types  and  coding  in  DBMS.  With  IMS/VS  1.1.5 
will  have  field  sensitivity. 

| 

B.  Logical 

Structure 

j 

1 

i 

. 

! 

Basic  unit  is  segment  but  with  1.1.5  programs  may 
retrieve,  insert,  replace,  by  fields.  Also,  logical 
structure  may  be  obtained  thru  secondary  indexing  or 
logical  relationships.  Logical  relationships  may  be 
!  between  segments  within  the  same  physical  database  or 
i  different  data  bases.  Structures  may  be  inverted  thru 
|  these  logical  relationships  or  secondary  indexing. 

!C.  Physical  I  Fixed  length  blocks.  Variable  length  records  and 
j  Structure  I  segments.  Corrmon  buffer  pool  stores  all  data  for 
|  |  OL/I  language  access. 


jo.  Access  Access  methods:  HSAM,  HISAM,  HIDAM,  HDAM.  HSAM,  HISAM 

|  Methods  j  are  sequential.  HDAM  and  HIDAM  are  direct.  HDAM  is 

j  randomized,  HISAM,  and  HDAM  support  the  inverted  file 
!  and  VSAM.  VSAM  can  be  used  for  HIDAM,  HDAM,  and  HISAM 

1  !  data  bases.  Inverted  data  bases  supported  by  all  of 

!  '  above. 


jE- 

1 

! 

i 

i 

1 

I 

Special  Storage 
Techniques 

1 

1 

i 

1 

Special  storage  techniques  in  HSAM,  HISAM,  HDAM,  and 

HIDAM  minimize  storage  requirements.  For  further 
data  compaction  an  exit  is  provided  in  DL/I  to  a  user 
routine.  VSAM  compacts  indexes.  Distributed  free 
space  can  be  requested  at  load  or  reorganization  time 
to  accommodate  insertion  of  segments  near  their  parents 
or  twins.  In  HIDAM  and  HDAM,  deleted  segment  space 
can  be  reused  for  new  data. 

b - - - - - - — ■ - 

II. 

i 

DATA  BASE  • 
CREATION  j 

User  program  normally  used  for  file  creation. 

III. 

DATA  BASE 
REVISION 

Logical  structures  are  modified  in  the  DBD  and  do  not 
necessarily  require  file  activity.  Experience 
indicates  minimal  impact  on  programs. 

Physical  structures  are  modified  in  the  DBD  and  will 
normally  require  dumping  and  reloading  of  the  DB. 

IV. 

DATA  A. 

MANIPULATION 

1 

Selection 

Level 

At  segment  level.  Items  can  be  described  by  names, 
codes,  or  relationship  to  other  items.  Can  be  made 
by  requesting  a  single  segment,  or  a  path  of  segments, 

|  or  in  1,1.5  by  retrieving  by  field.  Any  field  in  a 

1  segment  can  be  used  in  a  search  argument. 

i 

r 

Tl 
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IMS:  riOL-BASED  NON-CODASYL  DATA  BASE  MANAGEMENT  SYSTEM  (cont. 


FUNCTION 


PROPERTY 


parameters 


B.  Operators, 
Comparators, 
Logical 
Complexity 


Operators:  AND,  Oft,  LOGICAL  AND. 

Comparators:  EQ,  NOT  EQ;  GT,  GTEQ;  LT,  LTEQ. 
Limited,  heuristic,  and  special  structure  searches. 
Eight  logic  combinations  can  occur  at  each  segment 
level . 


C.  Reporting 


JSER 

INTERFACE 


B.  Mode  of 

Interaction 


I  Reporting  via  host  language  or  GIS.  G1S  produces 
I  default  reports,  page  numbering  and  multiple  page 
]  headers  automatically.  Output  to  all  devices  supported 
.  by  IMS  or  the  CPU.  A  response  can  be  reviewed  at  a 
i  terminal  and  then  sent  to  some  other  terminal  for 
!  further  processing/review. 


|  Through  service  calls  from  host  language. 
English-like  query  language.  (GIS) 

;  Batch  and  on-line.  Access  lockouts  at  the  page  level 
;  for  concurrent  update  purposes.  Concurrent  retrieval 
is  always  possible.  IMS/DS  provides  interminal 
communications  and  remote  job  control  with  dynamic 
priority  assignment.  Programs  are  not  locked  out, 
they  will  always  schedule  into  the  message  region(s) 
to  process.  Program  isolation  allows  two  or  more 
programs  to  operate  concurrently.  If  a  user  program 
!  updates  a  particular  segment,  no  other  program  can 
j  access  that  segment  until  the  update  program  reaches 
I  a  synchronization  point,  or  is  complete.  (Program 
|  Isolation  in  DC) . 


iC.  Error 
Messages 


Status  code  returned  in  response  to  all  requests  for 
data.  User  can  check  for  error.  Trace  facility 
can  be  invoked  at  test  time  to  provide  data  on  each 
DL/1  call. 

Malfunctions  and  errors,  displayed  at  the  IMS  master 
terminal . 


'D.  Documentation 


DI/1  general  information  manual  GH20-1260,  terminal 
operator  guide  SH20-9028,  system  program  reference 
manual  SH20-9027,  applications  program  reference 
manual  SH20-9026,  system  applications  design  guide 
SH20-9025,  utilities  reference  manual  SH20-9029, 
messages  and  codes  reference  manual  SH20-9030,  IMS/ VS 
conversion  planning  guide  SH20-9034,  systems  documenta 
tion  (licensed),  message  format  service  guide  SH20- 
9052,  and  advanced  function  for  comruni cations  SH20- 
9054.  GIS  general  information  manual  GH20-9035, 
executive  query  reference  guide  GH20-9043,  language 
reference  manual  SH20-9038,  MSG  and  codes  SH20-9039, 
advanced  query  reference  manual  SH20-9040,  program 
j  reference  manual  SH20-9037,  and  systems  documentation 
I  (licensed). 


IMS:  H0L-6ASED  NON-CODASYl  DATA  BASE  MANAGEMENT  SYSTEM  (cont.) 


FUNCTION 


VI.  APPLICATION 
PROGRAMMING 


PROPERTY 


A.  HOL  Interface 


B.  Subroutine 
Capabilities 


C.  Special 
Operations 


D.  I/O  Outside 
DMS 


PARAMETERS 


Standard  call  interface  specifying:  function,  logical 
file,  I/O  area,  search  argument.  Implemented  for 
COBOL,  PL/1,  ALC. 


Standard  host  language  rules  apply. 


None. 


Selected  sets  become  named  files  In  three  possible 
states:  a  vector  file,  an  ordered  list  file,  or  the 
data  file  itself.  Data  file  cag.  be  saved  on  any 
supported  device  via  the  DUMP  command.  Vector  file 
or  ordered  list  file  savable  in  multi  thread  version. 


DATA  BASE 
SECURITY, 
INTEGRITY, 
& 

AOMINIS- 


A.  Data 

Validation 


Checking  only  for  data  structure  and  sequencing. 
Exit  provided  for  user  program.  Editing  facilities 
in  query  language. 


B.  File 


I 


T RATI ON  j 

! 

i 

i 

1 

i 

Protection 

control-done  in  program  specification  block  (PSB). 

User  provided  encryption,  decryption  can  be  implemented 
within  the  DMS  through  a  special  exit.  Password  and 
user  profile  carry  security  to  the  field  level  and 
beyond  with  qualification  of  user.  Field  sensitivity 

In  I MS/ VS  1.1.5  (and  intent). 

i 

C.  Surveillance 

All  activities  logged,  including  security  violations. 

Logs  available  for  statistical  processing. 

D.  Failure 
|  Protection 

| 

System  automatically  logs  all  changes  to  any  data 
base  and  provides  complete  recovery  utilities  for 
restoring  data  bases  without  re-executing  application 
programs.  Checkpointing  and  restart  facilities 

Including  synch  of  DL/1  and  OS  checkpoints  and 
critical  areas  in  the  application  program  are  also 
provided.  System  can  continue  running  if  application 
program  fails. 

Table  C3-c 
HONEYWELL  6180/GC0S 

MRDS:  SELF-CONTAINED  OATA  BASE  MANAGEMENT  SYSTEM 


FUNCTION 


PROPERTY 


DATA  BASE  A.  Item 

DEFINITION  Description 


B.  Logical 
Structure 


[  C.  Physical 
!  Structure 


D.  Access  Methods 


|  E.  Special  Storage 
Techniques 


Naming:  1  to  32  character  names. 

Format:  Standard  PL/1  data  type  declarations. 
String  Types:  Fixed/varing  length  bit  and 
character  strings. 

Arithmetic  Types:  Real  or  complex,  fixed  or 
floating,  binary  or  decimal. 

Alignment:  Word  aligned,  byte  aligned,  or  un- 
all  gned. 

4  i  —  ■  ■  — — 

Groupings:  Data  base,  files,  relations,  tuples, 
attributes,  domains. 

Linkage:  See  "Physical  Structure." 

Structures:  Relational.  List,  tree,  network  struc¬ 
tures  definable  at  query  time. 

'  Data  Base:  Implemented  as  a  directory  and 
!  subordinate  files  in  the  Multlcs  Storage 

:  System. 

I  Disk  Assignment:  Interrelation  clustering  (op- 
I  tional),  fixed  and  variable  length  fields. 

!  Ordering:  Ascending  primary  keys, 
j  Linkage:  Direct  links,  secondary  Indexes. 

I  System  Interface:  Multlcs  virtual  file  manager 
!  (vflle-).  No  special  I/O. 

Methods:  Keyed  sequential,  random,  linked,  and/ 
or  hashed. 

I  Compaction:  Encode  and  decode  procedures. 

Variable  length  fields.  Unaligned  data. 
Efficiency:  Interrelation  clustering.  Blocked 
(pre-al located)  files  for  hashing.  Otherwise, 
keys  are  stored  as  B*-tree. 


II.  DATA  BASE  \ 

CREATION  ; 

1 

{ 

Creation:  "create-mrds-db"  Multlcs  command 
which  translates  a  user  written  data  model 
source  and  creates  a  corresponding  data 
base  shell. 

Loading:  Applications  program(s)  or  Linus  EUF 
"store"  request. 

•  ' 

III  DATA  BASE  ! 

Utilities:  'restructure  mrds  db '  Multlcs  com- 

REVISION 

mand  allowing  redefine,  define,  and  unde¬ 
fine  operations  on  files,  relations,  attri- 

- - -  ■— ! 

’ 

butes,  secondary  keys,  and  foreign  keys. 

Minimal  to  no  Impact  on  application  pro- 

-* 

grams  using  submodels. 

* 

•  ^ 
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HONEYWELL  6180/GC0S 

MRDS:  SELF-CONTAINED  DATA  BASE  (cont). 

-9 

FUNCTION 


PROPERTY 


PARAMETERS 


IV.  DATA 

MANIPULATION 


A.  Selection 
Level 


INTERFACE 


B.  Operators 
Comparators, 
|  Logical 

Complexity 


C.  Reporting 


A.  Manipulation 
Languages 


!  B.  Mode  Of 

Interaction 


C.  Error 
Messages 


D.  Documentation 


j  Selection:  attribute(s) ,  tuple(s),  relation(s). 

I  Qualification:  attribute(s) ,or  function(s)  of 
attribute(s). 

Comparators:”,  *  *,  >,  <,>«,<  «.  Arithme¬ 
tic  Operators:  +,-,*,/ 

Builtin  Scalar  Operators:  abs,  after,  before,  ceil, 
concat,  floor,  index,  mod,  reverse,  round,  search, 
substr,  verify. 

Built-inSet  Operators:  differ,  inter,  union.  Boolean 
Operators :  &  | ,  * 

!  Other  Operators:  User  definable  scalar  functions. 

|  Logical  Complexity:  Unrestricted.  Relational ly 
complete. 

j  *Linus  EUF  also  includes  the  builtin  set  operators 
;  avg,  count,  max,  min,  sum,  and  user  definable 
|  set  functions. 

i  Sorting:  Interface  to  standard  Multics  sort 
comnands. 

;  Reports:  Interface  to  the  Multics  Report  Program 
Generator  (MRPG) . 

;  *L1nus  EUF  contains,  in  addition  to  the  above,  a 
|  basic  report  capability  with  controllable  (or 
(  default)  headers  and  column-widths,  settable 
!  break-limits,  and  interfaces  to  the  Multics  File 
]  System  and  Lister  facility. 


HOL  relational  calculus  selection  expressions. 
Linus  EUF:  HOL  Sequel-like  selection  expres¬ 
sions. 


Interactive,  Absentee  (batch),  RJE;  Interface  at 
Multics  command  level,  Linus  EUF  subsystem,  or 
application  program  Call;  Concurrency  request 
thru  r/d/s/m  permission  at  data  base  or  relation 
level . 

Creation:  Compiler-like  error  messages  at  date 
base  and  data  submodel  creation  time. 
Application  Programs:  Symbolic  status/error 
codes  translatable  into  short  or  long  mes¬ 
sages. 

EUF:  Status/error  messages  within  Linus. 

MRDS  Reference  Manual  (AW53). 

Linus  Reference  Manual  (AZ49). 

MRPG  Reference  Manual  (CC69). 

Multics  "help"  command  and  Linus  "help" 
request.  Marketing  Education  F31  and  F32 
course  workbooks. 


Tao'.e  C3-6  ( Com. ) 

HONEYWELL  6180/GCOS 


MRDS:  SELF-CONTAINED  DATA  BASE  (cont.) 


FUNCTION 

PROPERTY 

W  ' 

PARAMETERS 

VI.  APPLICATION 
PROGRAMMING 

A.  Hoi  Interface 

Languages:  "call"  interface  from  all  Multics 
programing  languages.  Selection  expres¬ 
sion  Is  passed  as  a  character  string  argu¬ 
ment.  . . 

COBOL  OML  verbs  also  supported.  ® 

i  B.  Subroutine 
j  Capabilities 

|  t 

! 

Full  capability  to  store  procedures,  directly 
callable  from  comand  level  and/or  other 
programs  passing  arguments.  Recursion  and 
inter-language  calls  fully  supported.  Linus  EUF 
has  macro  storing  an  invoicing  capability. 

j  C.  Special 

Operators 

i  1 

<  1 

1  1 

MRDS  automatically  performs  data  conversions 
following  ANSI  PLI  conversion  rules. 

*L1nus  EUF  has  set  operators  avg,  count,  max, 
min,  sum  and  arithmetic  expressions  which 
operate  on  the  data  after  retrieval. 

!  D.  I/O  Outside  ! 

i  OMS 

j  : 

i  i 

Transportability:  Data  Is  completely  transport-  .  ' 

able  and/or  directly  usable  by  other  Multics  * 

facilities  such  as  the  graphics  system,  text  r  • 

formatter,  report  writer,  and  application 
programs. 

1  E.  Auxiliary 
;  Storage 

1 

1 

Temporary  Working  Areas:  Temporary  relations 

which  become  a  logical  (and  physical)  ex-  j.  _ 

tension  to  the  data  base  for  the  user  defln-  '  Ip'1 

Ing  them. 

Permanent  Working  Areas:  Standard  Multics  files. 

VII  DATA  BASE 
SECURITY, 
INTEGRITY, 

& 

ADMINIS¬ 

TRATION 

!  A.  Data 
!  Validation 

i 

i 

1 

Validation;  Domain  verification  enforcable  at 
store  and  modify  times. 

Integrity:  Encode  and  decode  normalization. 

Interrelation  integrity  enforceable  via  for-  — 

eign  key  concept. 

■ 

!  B.  File  Protection 

Level:  Access  rights  definable  at  data  base,  file, 
relationa  and  attribute  levels. 

Permissions:  Retrieve,  modify,  store,  delete 
permissions. 

Qualification:  Person  id,  project  Id  and/or  p 

global . 

Enforcement:  Hardware  and  software  enforce¬ 
ment  via  Multics  Access  Control  List  and/ 
or  ring  mechanism. 

C.  Surveillance 

Within  DBMS:  None  at  the  present  time. 

Outside  DBMS:  Standard  Multics  facilities  for 

auditing  access  violations. 
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< dD i 6  C3~o 
HONEYWELL  613G/GC0S 

__  _ MRDS :  SELF-CONTAINED  DATA  BASE  (cont.; _ 

FUNCTION  I  PROPERTY  j  PARAMETERS 

- j - - - - - 

0.  Failure  Backup:  Standard  Multics  backup  and  retrieve 

Protection  facilities  “dump  mrds  db"  command  to 

backup  (to  tape)  a  quiescent  data  base. 
Rollback:  Corrmitment/rollback  capability  (at 
file  manager  level)  is  cuurently  under  devel¬ 
opment. 

Restart:  Standard  Multics  Emergency  Shut- 
Down  (ESD)  and  restart  capability. 


TABLE  C3-7 
HONEYWELL  6180/GC0S 

MOOS:  Self-Contained  Data  Base  Management  System 


u 


FUNCTION 


I .  OATA  BASE 


PROPERTY 


A.  Item 

Description 


B.-  Logical 
Structure 


C.  Physical 
Structure 


i  D.  Access 
!  Methods 

| - 

E.  Special 
Storage 

j  Techniques 


DATA  BASE  A.  Selection 

MANIPULATION  ’  Level 


PARAMETERS 


1}  Data  -  Item  identifiers  can  consist  of  a  simple 
30  -  character  name  or  it  can  consist  of  that 
name  plus  an  entry  -  name  qualification,,  a 
mask  option,  and  or  a  conversion  subroutine 
specification. 

1)  Elements:  Data  (field),  record,  file,  data 
base;  schema;  networks  and  hierarchies. 

2)  The  Application  Definition  File  (ADF)  is  pre¬ 
pared  by  the  data  base  administrator  for  the 
MDQS  user.  The  ADF  contains;  data  base  re¬ 
ference  name,  entry  names,  and  item  names. 

3)  Relational  items 

1 )  The  CREATE  statement  createsone  or  more  new 
sequential  or  indexed  -  sequential  data  bases 
from  one  or  more  existing  (transaction)  data 
bases,  with  transformation  of  the  transaction 
entries  into  the  forms  predefined  for  the 
desired  new  data  base  entries. 

1)  Sequential,  index  sequential,  and  integrated 

2)  Concurrent  data  base  access 

1)  Implicit  storing  of  the  new  -  entry  data  base 
is  performed  only  if  the  CREATE  statement  is 
unlabeled. 

2)  In  explicit  storing  the  user  can  specify  a 
WHEN  SEQUENCE  error  to  do  additional  pro¬ 
cessing. 


1)  Data  base  creation  and  maintenance  permits 

a  user  to: 

e  access  data  with  full  concurrency 

e  create  a  data  base  from  one  or  more  tran¬ 
saction  data  bases 

a  update  multiple  data  bases  from  multiple 
transaction  files 

•  write  and  or  read  auxiliary  files  res- 
slding  on  disk  or  tape 

e  combine  two  or  more  data  bases  into  a  single 
data  base 

•  split  a  data  base  into  two  or  more  data 
bases 

•  create  a  data  base  that  is  a  subset  of  a 
data  base 


1)  At  the  element  level  for  Interactive  users 


VJ 


FUNCTION 


Table  C3-7 
HONEYWELL  5TSO/GCC3 
SELF-CONTAINED  DATA  BASE  '.DON: 


3aRAiME“ER$ 


8.  Operators, 

‘  Comparators, 

j  Logical 

1  Complexity 

!  Five  binary  operators,  unary  operator,  logical, 

;  relational.  Full  set  of  BOOLEAN  operators, 
i  Conditional  expression  comparators. 

! 

^  C.  Reporting 

•  -  j 

; 

•  ;  i 

\ 

1 

i  Flexible  Darameters  (or  default  specs)  for  page  w 

i  length,  indenting,  titles  etc.  Reporting  is  at 

J  the  Query  level.  Defaults  or  user  specifies 
|  titles,  column  separators,  and  data  item  display 
|  editing  characters  with  clauses  that  serve  as 
!  modifiers  to  the  PRINT  statement.  These  clauses 

I  are  TITLE,  COLUMN,  and  PRINT. 

V.  USES 

INTERFACE 


A.  Manipulation 
Language 


!  B.  Mode  of 
j  Interaction 

•  C.  Error 
Messages 

!  0.  Documentation 


The  conversational  Management  Data  Query  (CMDQ) 
subsystem  through  a  conversation  with  the  terminal 
user,  generates  a  MDQS  procedure  which  will  access 
a  data  base  and  display  the  desired  information 
at  the  terminal  or  optionally  on  a  file  for  later 
viewing.  The  Query  Language  allows  a  user  to 
generate  a  report. 

Primarily  procedure  selection. 

On-line  and  batch. 


The  user  is  given  a  list  of  the  valid  responses. 


Standard  references . 


F 

'll.  APPLICATION  i 

No  HOL  interface. 

PROGRAMMING 

- i - 

- - - 

DATA  BASE 
SECURITY, 
INTEGRITY , 

& 

trMiwrs- 

i  A. 

Data 

Validation 

TRATION 

i 

i  B. 

i 

File 

Protection 

C. 

Surveil lance 

•  0. 

Failure 

Protection 

Data  value  integrity  is  user's  responsibility.  The 
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SECTION  0-1 
PURPOSE 


A  survey  of  software  tools  on  the  candidate  target  environments  was  conducted 
during  the  first  phase  of  the  AMT  contract.  The  purpose  of  the  survey  was  to 
identify  software  tools  that  could  be  incorporated  in  the  AMT.  The  survey  was 
limited  to  the  candidate  environments  because  it  was  felt  it  was  beyond  the 
scope  of  this  effort  to  transport  tools  from  other  environments.  The  criteria 
for  selection  of  a  tool  for  consideration  for  incorporation  in  the  AMT  were: 

o  Applicability  to  software  measurement  (Did  the  tool  provide  any  metric 
data?) 

o  Portability  of  tool  (Can  the  tool  be  used  on  different  hardware 
conf igurations?) 

o  Interoperability  of  the  tool  (How  many  modifications  to  the  tool  are 
necessary?) 

o  Usability  of  the  tool  (How  much  effort  is  required  to  learn  how  to 
operate  the  tool?  How  much  effort  is  there  to  preparing  input  and 
interpreting  output  that  was  tool-driven?) 

The  results  of  this  RADC  Tool  Survey  are  presented  In  matrix  form  In  paragraph 
3.  Background  information  and  analysis  of  the  state-of-the-art  of  software 
tools  and  their  applicability  to  metric  appear  in  paragraph  2,  preceding  the 
RADC  Tools  Survey.  As  a  result  of  tis  analysis,  selective  RADC  tools  that 
have  compatible  hardware/operating  systems  with  the  target  environments  are 
also  included,  in  the  matrix  of  paragraph  3.  Finally,  paragraph  4  describes 
the  actual  tools  to  be  used  In  the  AMT,  what  other  tools  were  considered  for 
use,  or  what  tools  were  applied  during  its  development. 


SECTION  D-2 

CODING  AND  IMPLEMENTATION:  METRICS  APPLICABILITY 


The  origin  of  code  inspection  was  structured  programming  and  allied  software 
engineering  technologies  of  the  early  1970's.  The  goal  of  automated  static 
analysis/evaluation  has  been  to  automate  the  compliance  with  the  techniaues 
and  make  a  search  of  program  properties. 

The  program  parameters  are  structure-based  (program  logical  and  data 
structure,  naming  conventions,  documentation  conventions,  etc.  ),  control/data 
flow  based  (avoidance  of  undue  control  complexity;  assurance  of 
well-definedness  of  variables,  etc.),  and  interface  based  (assurance  of 
correspondence  between  modules,  subsystem,  inter-system,  etc).  The 
anomaly-detecting  metrics  have  to  do  with  standards  enforcement  (deficiencies 
in  source  code),  whereas  the  predictive  metrics  quantify  the  logic  of  design 
and  implementation. 

For  example,  the  JOVIAL  Automated  Metric  System  (JAMS)  is  designed  to  collect 
structural  information  about  JOVIAL  programs.  GE' s  Integrated  Software 
Development  System  (ISDS)  provides  a  capability  to  analyze  other  languages 
including  FORTRAN  PDL,  IF TRAN,  and  PASCAL.  A  major  subsystem  of  ISDS,  the 
generalized  parser  (GNP),  the  grammar  description  language  (GDL)  and  grammar 
tables,  provides  this  capability  and  will  be  used  in  the  AMT. 

Symbolic  evaluation  of  code  has  as  its  goal  the  "interpretation"  of  program 
behavior  at  the  programming  language  level.  Assumption  must  be  made  about  the 
environment,  the  deterministic  properties  of  the  programming  language 
behavior,  and  the  outcome  of  symbolic  execution  results.  On  systems  such  as 
DISSECT  or  MACSYMA  the  user  Interactively  chooses  a  path  and  performs  symbolic 
interpretation  of  actions  along  the  chosen  path.  The  system  then  displays  the 
"formulas"  to  the  user.  The  user  compares  original  and  implemented  formulas 
for  equality.  Differences  between  computed  and  actual  formulas  are  mistakes. 
Special  formula  formatting  methods  are  used  to  make  these  differences  highly 
visible.  Final  control  software  Is  not  yet  available.  Symbolic  evaluation 
has  good  candidate  potential  for  the  accuracy  metrics  at  the  system  level. 
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The  final  type  of  static  analysis  tools,  proof  of  correctness,  can  be  used  at 
the  system  level,  subsystem  level,  or  the  module  level  as  assessments  of 
different  levels  of  correctness.  The  Failure  of  Proof  Method  (FPM),  uses  a 
mathematical  approach  to  proving  the  correspondence  between  a  program  and  its 
formal  specification.  The  consistency  metric  is  highly  visible  here. 

Dynamic  testing  is  achieved  through  system  exercising  of  programs.  Typical 
self-testing  metrics  for  higher  level  language  systems  have  been  built  on  a 
experimental  basis  and  include: 

o  Automatic  specified  percentage  of  program  logical  segment  coverage  in 
any  one  test;  aggregated  test  coverage  of  close  to  100%. 

o  Assistance  in  setting  input  values  and  evaluating  output  values. 

o  Some  form  of  automated  results  comparison. 

These  dynamic  test  tools  consist  of  two  basic  modules,  an  instrumentation 
module  and  an  analyzer  module.  The  source  language  program  is  submitted 
directly  to  the  instrumentation  module.  Then  the  instrumentation  module 
accepts  the  source  program  of  the  module  under  test  and  instruments  it  by 
inserting  additional  statements  in  the  form  of  counters  or  sensors.  The 
instrumented  source  file  is  compiled  and  executed.  At  this  point  an  analyzer 
module  produces  a  report  documenting  the  behavior  under  the  test  during  its 
execution. 


Typical  metric  -  like  data  reported  are: 


o  Max  and  min  values  of  variables, 
o  Number  and  percentage  of  subroutine  calls  executed, 
o  Measures  of  program  complexity, 
o  Statement  consistency  checks, 
o  Program  cross-references, 
o  Trace  capability, 
o  Flagging  of  non- ANSI  code, 
o  Logically  impossible  -  path  detection, 
o  Subroutine  argument/parameter  verification, 
o  Data  range  check, 
o  If  statement  trace, 
o  Branch  trace, 
o  Subroutine/statement  timing 
o  Min/max  assignment  values, 
o  First/last  assignment  values, 
o  Min/max  DO  Loop  Control  Variable, 
o  Final  DO  Loop  Index  Value, 
o  Final  branch  values. 

o  Statement,  path,  segment,  module  interface  or  flow  execution  frequencies 
o  Specific  data  associated  with  each  executable  source  statement, 
o  Subroutine  retrace  capability,  complete  calling  tree,  reverse  execution 
capability. 

o  Performance  indices  for  modules  and  input  data. 

A  list  of  dynamic  tools  would  include:  JAVS,  CABS,  FAVS,  RXVP,  FORTUNE,  CIP, 
FORSAP,  FETE,  PPOGFORT,  PROGTIME,  TPL,  and  TAP. 

The  goal  of  mutation  analysis  is  to  show  that  small  changes  in  program  are 
discovered  by  test  data.  Conversely,  the  test  data  must  be  strong  enough  to 
catch  the  significant  errors.  Relevance  to  error  detection  metrics  is  obvious. 
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The  Pilot  Mutation  System  (PIMS)  has  been  applied  to  FORTRAN  and  COBOL  pilot 
systems.  Magnitude  of  the  mutant  error  Is  classified  as: 

o  Program  does  not  compute. 

o  Program  computes  but  does  not  run  test  data. 

o  Program  compiles,  test  run  is  satisfactory,  and  the  program  is  either 
logically  equivalent  to  the  original  or  test  data  is  not  good  enough. 

Reliability  analysis  is  still  in  Its  infancy.  The  goal  Is  to  determine 
whether  all  defects  have  been  reliably  removed  by  tests.  Any  error  must  be 
made  known  by  some  combination  of  inputs.  Following  this  theoretical  approach 
of  examining  all  possible  input  combinations  is  prohibitive  in  terms  of  cost 
effectiveness  and  computer  time /capacity.  The  Next  Error  Discovery  Predltion 
method  fails  because  software  reliability  simply  does  not  follow  the 
probability  laws  of  hardware  reliability. 


SECTION  D-3 

MATRIX  OF  SOFTWARE  TOOLS 


The  matrix  of  software  tools  having  potential  metric  applicability  follows  In 
Figure  03-1.  It  includes  tools  currently  in  use  or  planned  for  at  RAOC  and 
additional  non-RADC  tools  also  worthy  of  consideration  for  AMT  development  or 
usage.  Figue  03-2  illustrates  the  Software  Tools  Survey  Sheet  used  to  collect 
information  about  the  target  environment's  software  tools. 
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Figure  D3-1  (Continued) 


Figure  D3-1  (Continued) 


Figure  D3-1  (Continued) 
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SECTION  0-4 
TOOLS  USED 

l 

Tne  tools  used  in  AMT,  considered  for  use,  and  applied  during  the  development 
of  the  AMT  are  identified  in  Table  04-1.  The  tools  identified  as  used  in  the 
AMT  were  actually  incorporated  in  the  software  as  part  of  the  system.  The 
|  tools  identified  as  considered  for  use  are  candidates  for  interfacing  with  the 

AMT.  This  was  not  done  because  these  systems  were  not  available  during  the 
span  of  the  project.  The  last  category  of  tools  identified  are  those  tools 

used  on  the  AMT,  ie.  these  tools  were  utilized  by  the  development  team  during 

j  the  development  of  AMT.  SPOL  is  a  program  design  language  with  ADA-like 

*  constructs  and  concepts.  The  design  was  written  in  this  language  and  some 

metrics  automatically  applied  by  the  Integrated  Software  Development  System 
■ISOS).  The  implementation  language  utilized  was  IFTRAN,  a  structured  FORTRAN 
preprocessor  developed  by  General  Research  Corporation. 
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MISSION 

of 

Rome  Air  Development  Center 

RA PC  plan*  and  execute*  researc h,  development,  test  and 
detected  acquisition  programs  in  support  of  Command,  Control 
Communications  and  Intelligence.  [C3 1)  activities.  Technical 
and  engineering  Support  within  areas  of  technical  competence 
is  provided  to  ESD  Program  Offices  (PO*)  and  other  ESP 
element*.  The  principal  technical  mission  areas  are 
communications,  electromagnetic  guidance  and  control,  sur¬ 
veillance  of  ground  and  aerospace  objects,  intelligence  data 
collection  and  handling,  information  system  technology, 
ionospheric  propagation,  solid  state  sciences,  microunve 
physics  and  electronic  reliability,  maintainability  and 
compatibility. 


